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Space Station Human Factors Research Review 


PREFACE 


This conference proceeding is a compilation of the papers presented at the Space Station 
Human Factors Research Review held at NASA Ames Research Center from Decem- 
ber 3-6, 1985. These presentations represent the first year of research supported by the 
Space Station Advanced Development program as well as on-going related research sup- 
ported by other NASA programs. 

Each day of this research review was dedicated to a different focus or discipline. The foci 
represent the various areas of expertise in the Space Human Factors Office and the Aero- 
space Human Factors Research Division at Ames Research Center. In general, the structure 
of the conference was to proceed from the more general topics to the more specific issues 
during each day and throughout the week. 

Vic Vykukal, a specialist in advanced space suit design, chaired the first day’s session, 
EVA Research and Development. After Vykukal presented an introduction to EVA 
Research and Development at Ames, representatives of each of the three aerospace contrac- 
tors participating in the EVA Systems Study presented their views on Implications for 
Man-System Design. The final presentation related experiences in the deep-sea diving indus- 
try that are relevant to EVA. 

Yvonne Clearwater, an environmental psychologist who is pioneering the quantitative 
modeling of human spatial habitability, chaired the second day, Space Station Habitability: 
Behavioral Research. After Clearwater presented an introduction to the Space Station 
Habitability Research Program within the Space Human Factors Office, contractors and 
grantees made presentations on habitability, productivity, operational simulation and 
aesthetics for space station design guidelines. The session concluded with a panel discussion 
consisting of the principal speakers. 

Marc Cohen, an architect in innovative Space Station design, chaired the third day. Space 
Station Habitability and Function: Architectural Research. After Cohen presented an 
introduction to Ames Research Center Space Station Architectural Research, each of the 
contractor or grantee architects presented reports on the progress of their work in architec- 
tural design research. The session concluded with a panel discussion consisting of the princi- 
pal speakers. 

Trieve Tanner, Acting Assistant Chief for the Research for the Aerospace Human Factors 
Research Division, chaired the fourth day, Inhouse Advanced Development and Research. 
After Tanner gave a brief introduction, the members of the division’s basic research disci- 
pline groups presented papers in their respective areas of expertise: Cognition and Percep- 
tion, Workload and Performance, and Human/Machine Integration. 

Each of these four sessions is published as a separate volume of NASA CP-2426, with each 
day corresponding to the sequentially numbered volume. 
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EXECUTIVE SUMMARY 


Volume IV completes the review of our Space Station Human Factors Program. 
Volume I examined the EVA-oriented portion of the program. Volumes II and III dealt 
with the research that is associated with habitability - making the total habitable space on 
Space Station supportive of human well being and productivity. 

The research that will be presented in this volume is more specifically associated with 
the human factors of the work station. Much of this work had its origin in addressing human 
factors issues in an aeronautical work station - the cockpit. Where we have seen analogs 
between the cockpit as a work station and the potential work stations on the Space Station, 
we have focused elements of our program on Space Station issues. 

The work station that we will be simulating in our first mockup, discussed in 
Volume II, is a proximity-operations work station. Proximity operations will involve some 
of the more important activities of at least the initial Space Station, Shuttle docking, satel- 
lite servicing, and other initial work. 

A proximity-operations work station can place great burdens on the crew operators if 
not properly designed. Yet proper design of the various interfaces between crew and 
machine can enhance productivity by avoiding the limitations and taking advantage of the 
capabilities of both the human and the machine. Some of the reports then will be focused 
on proximity operation issues; others will be focused on issues associated with other Space 
Station work stations. Two of the tasks that we are reporting today, workload and perfor- 
mance, and perspective displays (and all of the tasks reported in the other volumes) have 
been supported as part of the Space Station Advanced Development Program. The remain- 
der of the tasks (and some of those reported in the other volumes) are part of the Office of 
Aeronautics and Space Technology’s Research and Technology program. 

The reports represent the Space-Station-oriented portions of the programs of three 
Working Groups within the Aerospace Human Factors Research Division: the first group is 
concerned with human factors issues associated with human perception and cognition; the 
second group with human workload and performance; and the third group with specific 
human/machine interaction issues. Each set will be introduced by the group leader. 
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Space Station Proximity Operations 
and Window Design 


Richard F. Haines 

Aerospace Human Factors Research Division 
Ames Research Center - NASA 
Moffett Field, Calif. 94035 

March 24,1986 


ABSTRACT 

On-orbit proximity operations (PROX-OPS) refers to all EVA within one km of the 
Space Station. Because of the potentially large variety of PROX-OPS, very careful planning 
for Space Station windows is called for and must consider a great many human factors. This 
paper reviews some of these human factors using as its outline a NASA Technical Memoran- 
dum in preparation by the author. The following topics are discussed: (l) basic window 
design philosophy and assumptions, (2) the concept of the local horizontal - local vertical 
on-orbit, (3) window linear dimensions, (4) selected anthropomorphic considerations, (5) 
displays and controls relative to windows, (6) full window assembly replacement, (7) Sum- 
mary and Conclusions, and (8) References. 


INTRODUCTION 

Relatively little has been written on the important subject of Space Station windows. 
A NASA Technical Memorandum (TM) now in preparation (Haines, 1986) documents most 
of the prior technical references dealing with the optical, geometric, and structural properties 
of windows installed on prior US and Soviet spacecraft and will not be repeated here. Good 
window designs result from a deliberate and painstaking analysis of all of the tasks which the 
viewer must carry out through the window(s), the operational capabilities and limitations of 
the entire Space Station on-orbit, the perceptual and physiological capabilities and limita- 
tions of the viewer over time, and a host of other factors too complex to deal with here. As 
stated in NASA report JSC-19989 (1984; pg. 3) describing the so-called Reference Configura- 
tion for the Space Station, ,l One of the principal advantages of this configuration is the good 
viewing afforded to all payloads, both externally-mounted and internally mounted." Such 
viewing will require properly designed windows. 

Figure 1 presents the Table of Contents for the forthcoming TM by the author with a 
mark at those subjects that are discussed (briefly) here. 
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Figure 1 


Table of Contents of NASA TM Entitled 

“Space Station Proximity Operations Windows: Human Factors Design Guidelines” 

(Haines, 1986) 


0.0 Introduction 

0.1 Justification for this Report 
0.2 Subjects Not Covered 
0.3 Window Design Assumptions 
0.4 Window Design Philosophy 
0.5 Proximity Operation Station 

1 .0 Solar Illumination Ambient Geometry and Cyclic Timing 

1.1 Basic Characteristics 

1.1.1 Radiation and Photometric Constants 

1.1.2 Angular Constants 

1.2 Temporal Characteristics 

1 .2.1 Orbital Inclination Parameters 

1 .2.2 Orbital Altitude Parameters 

1 .3 Earthlight and Moonlight 

2.0 Artificial Illumination Geometry 

2.1 Space Station Light Sources 

2.1.1 Exterior Mounted Floodlights and Spotlights 

2.1.2 Exterior Mounted Running Lights 

2.1 .3 Interior Mounted Lights 

2.2 Shuttle (or Moving Target) Light Sources 

2.2.1 Exterior Mounted Floodlights and Spotlights 

2.2.2 Exterior Mounted Running Lights 

2.2.3 Interior Mounted Lights 

2.2.4 Payload Bay Floodlights 

3.0 Human Observer Tasks and Related Window Design Requirements 

3.1 Selected “Return/Rendezvous” Maneuver Requirements 

3.1.1 Manned Active Approaches (±) 

3. 1.1.1 Local Vertical Local Horizontal Radius Vector 

3. 1.1.2 Local Vertical Local Horizontal Momentum Vector 

3 . 1 . 1 .3 Local Vertical Local Horizontal Velocity Vector 

3.1 .2 Unmanned Approach to Manned Space Station 

3.2 Special “Deployment/Separation” Maneuver Requirements 

3.3 Space Station Construction Activities 

3.4 Window Field of View-Related Human Factors Design Tradeoff Parameters 

3.4.1 Window (Linear) Dimensions 

3.4.2 Eye-Inner Pane (Set-back) Distance 
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Figure 1 — Continued 


3. 4 .2.1 Anthropomorphic Considerations 

3 .4.2.2 Surface Contamination Considerations 

3.4.3 Design Eye Point Lateral Offset from Window Centerline 

3.4.4 Allowable Head/Eye Movements 

3. 4 .4.1 Radial (Set-back) Translation 

3. 4 .4.2 Vertical Translation 

3. 4 .4 .3 Horizontal and Vertical Eyeball Rotation Limits 

3.4.5 Gap Width Considerations 

3.4.6 Multiple, Adjacent Windows 

3.5 Distribution of Various Sensitivities in the Visual Field 

3 .6 Number of Viewers per Window 

3.7 Number of Windows and Location Relative to Viewers 

3.8 Prox-Ops Station Controls and Displays Relative to the Window(s) 

4.0 Window Optical Requirements 

4.1 Transmissivity 

4.1.1 Absolute (Visible Spectrum Near IR-Near UV) 

4.1 .1 .1 Prior U.S. Spacecraft Specifications 

4.1 .1 .2 Selected Visual Functions Requiring Certain Wavelength 
Ranges(s) 

4.1.2 Spectral (Selected Wavelengths) 

4.1 .2.1 Dichroic Filters for Heat Rejection/Control 

4.1 .2.2 Monochromatic Filters for Contrast Optimization, Scientific 
Tasks 

4.1.3 Variability 

4.2 Electrically Conductive Optical Coatings 

4.3 Line of Sight Deviation (Prism Effect) and Light Scatter 

4.4 Line of Sight Displacement 

4.5 Magnification/MiniFication 

4.6 Astigmatism 

4.7 Surface Reflections and Haze 

4.7.1 From Exterior Light Sources 

4.7.2 From Interior Light Sources 

4.7.2. 1 Moving (Illuminated) Surfaces/People 

4. 7.2.2 Fixed Light Sources 

4.7.3 Multiple Reflections 

4.8 SunShades 

5.0 Window-Maintenance and Protection 

5.1 Types of Degradation/Damage 

5.1.1 Surface Contamination (Cleanable) 

5. 1.1.1 Exterior 

5. 1.1. 2 Interior 

5. 1.1. 2.1 Body/Skin Oils 

5. 1.1. 2.2 Breath Condensation 

5.1.2 Surface Contamination (Non-Cleanable) 

5. 1.2.1 Exterior 

5. 1.2. 1.1 Rocket Plume Impingement (Solid, Liquid) 

5. 1.2. 1.2 Other Sources 
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Figure 1 - Concluded 


5.1.3 Permanent Surface Damage (Requiring Replacement) 

5 . 1 .3 . 1 Micrometeorite Impact (Pitting) 

5.1 .4 External (Pressure Pane) Breakage 

5.1.5 Puncture of All Window Panes (Pressure Loss) 

5.1.6 Spectral Transmission Change (Randiation/Thermal) 

5.1 .7 Thermal Shock Breakage 

5.1.8 Other Causes of Degradation 

5.2 Candidate Methods of Maintenance 

5.2.1 Hand Cleaning 

5 .2.2 Mechanical Polishing/Buffing 

5.2.3 Pane Replacement 

5.2.4 Full Window Assembly Replacement 

5.2.5 Replaceable Shields 

5.3 Candidate Methods of Protection On-Orbit 

5.3.1 Shutters (Mechanically Actuated) 

5.3.2 Laminar Air Flow 

5.3.3 Permanent Temporary Frames 

5.4 Crew Safety Threats Related to Prox-Ops Windows 

5.4.1 Catastrophic Failure 

5.4.2 Hazardous Failure 

5.4.3 Inconvenience Failure 

6.0 References 

7.0 Appendices 

A. Mercury Capsule Windows and Related Details 

B. Gemini Capsule Windows and Related Details 

C. Apollo Command Module Windows and Related Details 

D. Apollo Lunar Excursion Module Windows and Related Details 

E. Skylab Windows and Related Details 

F. Space Shuttle Vehicle Windows and Related Details 

G. Soviet Vostok Capsule Windows and Related Details 

H. Soviet Voskhod Capsule Windows and Related Details 

I. Soviet Soyuz Capsule Windows and Related Details 

J. Soviet Salyut Vehicle Windows and Related Details 
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L BASIC WINDO W DESIGN PHILOSOPHY AND ASSUMPTIONS 

The overall design philosophy for PROX-OPS windows include the following elements: 
(a) all windows shall support the greatest degree of external and internal situational aware- 
ness as possible, (b) all windows shall provide the greatest level of bodily protection possible 
from external radiation sources, and dangers resulting from changes in pressure, temperature, 
etc. (c) each window shall not allow visual degradation to occur due to veiling glare, flash 
blindness, or other unexpected luminous event during critical operational periods, and (d) 
each window shall provide as large a horizontal and vertical field of view (FOV) as possible. 

Several basic assumptions have been made which take into account other engineering 
and mission requirements presented elsewhere (Donahoo and Anderson, 1985; Mcdonnell 
Douglas Astronautics, 1985; Oberg, 1982). They include: (a) for most PROX-OPS out-the- 
window activities there will be only one viewer per window, (b) a maximum window dimen- 
sion of 20 inches will be allowed. Nevertheless, it is reasonable to assume that a 20” x 30" 
(50.8 x 76.2 cm) rectangular window will be permitted from a structural design standpoint, 
(c) round windows will not be used for PROX-OPS (primarily) because such a window shape 
eliminates any (target vehicle) roll cues or body orientation cues for the viewer, (d) window 
panes will be flat glass with an inert, dry gas filling inner cavities, (e) the thickness of win- 
dow frames surrounding each window will be as small as possible to reduce visual occlusion 
of external objects, (f) all windows that are to be used for making color discriminations shall 
possess neutral spectral transmission so that perceived target object hues are not altered, and 
(g) each window shall be designed to accommodate a 'design eye volume” (DEV) of approxi- 
mately 0.6 cubic meter centered on the center of the window and set back 12” (30.5 cm) 
from the inner most pane. 


//. THE LOCAL HORIZONTAL - LOCAL VERTICAL CONCEPT 

OF ON-ORBIT SPACE STATION STABILIZATION 

Figure 2 illustrates the local horizontal-local vertical (LH-LV) concept of Space Station 
stabilization on-orbit and related nomenclature. The particular configuration of modules 
shown is not important. Throughout its orbital travel, the Space Station will pitch so as to 
maintain the center of the earth directly below it. 

Figure 2 

Local Horizontal - Local Vertical Space Station Stabilization 
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For certain approach trajectories (e.g., the minus R bar shown in Figure 3), the 
approaching target vehicle will not be visible from windows located in the end-cap of a 
module that are facing only in the minus V bar direction until the vehicle is very near the 
Space Station [typically under 100 ft (31 m)]. The windows must accommodate a large vert- 
ical FOV for this type of approach maneuver. 

Figure 3 

Nominal Minus R bar PROX-OPS Approach Trajectory 



For other approach trajectories such as the plus V bar shown in Figure 4, the PROX- 
OPS windows must face in the direction of the velocity vector looking in the general direc- 
tion of the rising sun. This will place the approaching target vehicle between the sun and 
the viewer and create many practical problems of target visibility along with optical design 
problems. 


Figure 4 

Nominal Plus V bar PROX-OPS Approach Trajectory 
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Ill . WINDO W LINEAR DIMENSIONS 

The geometric variables which will determine the total available FOV of a window are 
shown in Figure 5. Figures 6 and 7 present the total FOV angle for a 9 M (23 cm), 18" (46 
cm), and 48" (122 cm) wide window as a function of the lateral offset distance (X - Xl in 
Figure 5) for a 6" (5 cm) and 18" (46 cm) set-back distance. Clearly, both set-back distance 
and lateral offset play crucial roles in limiting the available FOV. When anthropomorphic 
considerations are taken into account, nominal window sizes may be determined (cf. Figure 
8) which will then permit operational planners to know, in advance, whether an approaching 
vehicle following a particular trajectory will remain visible in a given window or not and at 
what point in its approach will it first appear. Such prior knowledge is very useful. 

Figure 5 

Geometric Variables Which Determine the Visual Angle 
for a Single Window 



Figure 6 

Visual Angle (deg.) for Three Window Sizes as a Function 
of Lateral Offset (in.) for a Six Inch Eye Set-Back Distance 
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Figure 7 


Visual Angle (deg.) for Three Window Sizes as a Function 
of Lateral Offset (in.) for an 18 Inch Eye Set-Back Distance 



Figure 8 

One Viewer Per Window Spacing Recommendation 
for a Small Set-back Distance 
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The last geometric variable considered here in regard to the maximum achievable FOV 
through a given window is that of bezel thickness, i.e., the thickness of the wall in which a 
window is installed. In Figure 5 this is shown (for convenience) as being equivalent to the 
separation distance between the innermost and outermost window panes. Calculations have 
been made of the visual angle across a 12" (30.5 cm) wide window with the eye on centerline 
but set-back various distances. Figure 9 presents the results of such calculations as a func- 
tion of bezel thicknesses from zero to 5" (12.7 cm). It may be noted that visual angle 
through the window increases with decreasing set-back distance and also with decreasing 
bezel thickness in a regular fashion. Such data may be used to perform engineering trade-off 
studies of various geometric designs. 


Figure 9 

Visual Angle (deg) for Various Set-back Distances and Bezel 
Thicknesses for a 12" (30.5 cm) Wide Window 



IV. SELECTED ANTHROPOMORPHIC CONSIDERATIONS 

Among the many anthropomorphic considerations related to Space Station window 
design are those related to micro-gravity body posture (Griffin, 1978; Jackson et al., 1975). 
The basic situation is illustrated in Figure 10 for a set-back distance of 12" (30.5 cm) from 
the innermost pane. Because the head flexes forward, the average line of sight (LOS) is 
depressed by about 20 to 30 deg arc compared to its nominal direction in one g. The legs 
and arms also tend to bend somewhat as shown. 
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Figure 10 

Approximate Neutral Body Position in Prolonged Weightlessness 


FORWARD HEAD 
TILT ONLY 

T 2 Tj 

/ HEAD TILT BACK 


i 
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When the viewer must look through a Space Station window for prolonged periods of 
time using relatively small set-back distances and (hand, knee) body clearance (cf. Figure 11), 
the body axis angle T-O-P must approach 180 deg rather than flex forward to about 150 deg. 
This can result in neck tension, pain and fatigue. By increasing the eye set-back distance, 
the body can assume a more natural and comfortable posture as is shown in Figure 12. 

Figure 11 

Illustration Showing the Deviation from the Neutral Body 
Position Required to View Through a Window 
that is 12 Inches Away 
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Figure 12 


Illustration Showing the More Natural and Comfortable 
Body Position Possible by Increasing the 
Eye to Window Set-back Distance 



V. DISPLAYS AND CONTROLS RELATIVE TO WINDOWS 

The human factors specialist familiar with traditional aircraft cockpit layout and design 
should be consulted in regard to how best to locate informational displays and controls in a 
PROX-OPS station. A fundamental difference between the two design environments is the 
fact that, on-orbit, the design eye point turns into a design eye volume . Due to the fact that 
viewers in prolonged micro-gravity will not need to be as physically constrained as they are 
in the cockpit of an airplane, they will necessarily experience more (voluntary and involun- 
tary) slow head translation per unit time. For most extra-vehicular observing tasks this 
should not prove to be a problem. For many interior observing tasks, such as monitoring a 
precision TV display during the final stages of berthing, head translation may lead to some 
serious perceptual and operational problems. 

Another consideration with regard to the proper layout design of PROX-OPS displays 
and windows has to do with maintaining as much relevant display information as possible 
within the viewer’s binocular visual Field while he or she is viewing out a window. This con- 
cept is illustrated in Figure 13. A design eye volume of approximately 0.6 cubic meters is 
recommended for each window such that the viewer could move any place within this 
volume and be able to perceive the same quantity and quality of interior (panel) information. 
Such a design requirement will help the viewer maintain a high degree of situational aware- 
ness. Space does not permit a fuller treatment of this topic. The interested reader should 
consult Haines (1975) and its references for further information. 
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Figure 13 


Proximity Operations Control Station Concept to Achieve 
Full Binocular Visual Sensitivity While Viewing 
Through a Given Window 



Locating PROX-OPS displays and controls must take into account the location, spac- 
ing, shape, number, and size of the windows present. A candidate window arrangement that 
is being evaluated at Ames in the Proximity Operations Simulator consists of five windows 
arranged in an inverted T. Figure 14 shows this arrangement. 

Figure 14 

Front View of a Scale Model Ellipsoidal End Cap Designed by Marc M. Cohen, AIA, 
Showing a Candidate 'Inverted T" Arrangement of Windows. 
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This arrangement of five windows is repeated 90 degrees to the left and 90 degrees to the 
right as well. This end-view photograph is of a scale model module end-cap with an off- 
center berthing port shown near the bottom. This arrangement of 15 separate windows offers 
a number of advantages which include: (l) overlapping fields of view by two or more crew 
who are viewing through different sets of windows, (2) wide field of view in both the horizon- 
tal and vertical directions simultaneously through any single set of five windows, and (3) use 
of standardized window assemblies in at least three of the five window locations. This win- 
dow layout will also allow for radial rib construction radiating from a common center. 

Figure 15 is a photograph taken inside the Proximity Operations Simulator at Ames 
showing the initial layout and construction of the windows. The following window design 
features may be noted with regard to each grouping of five windows: (1) The design eye 

volume is approximately 0.6 cubic m which permits a maximum horizontal Field of view of 
125 degrees arc and a maximum vertical field of view of just under 100 degrees. Excellent 
external situational awareness is ensured by this layout. (2) The two lateral windows flank- 
ing the center window are mirror images of each other in shape. (3) The two upper-most 
windows are identical. 


Figure 15 

Photograph of Full Scale Mockup During Construction 
Showing Array of Five Prox-Ops Windows 



(4) All window frames are smallest at the outer glass surface, i.e., the frames are larger on 
the observer’s side than on the space side. This ensures that the observer will always know 
that the maximum external scene is visible without having to move the head to make sure. 

(5) The top and bottom (horizontal) window frames are all parallel with the local horizontal 
of the Space Station. This features aids in making judgments of the horizontality of 
approaching target vehicles when necessary. (6) The height of the center window is slightly 
raised relative to the height of the left- and right-hand windows which permits greater 
down-looking capability to each side. (7) All window frames are painted with a medium 
reflectance (approx. 65 percent) flat grey to ensure a low contrast window frame surround at 
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all times, i.e., the observer will always be able to see the window frame and use it to assist in 
making relative target motion judgements. (8) As many as five persons can view out of each 
group of five windows by assuming different body orientations relative to each other. As 
many as three persons can view out of each group of five windows when their body axes are 
parallel and their feet are in the same direction. 

Other PROX-OPS station viewport concepts have been proposed by Bell and Trotti 
(1985) using a module connecting node as the location for viewing windows. They are repro- 
duced in Figures 16 through 18. Each succeeding figure presents increasing FOV and exte- 
rior situational awareness. The hemispheric viewport proposed in Figure 18 appears to be 
beyond the current level of optical material processing in terms of keeping line of sight dis- 
tortions at any penetration point to an acceptable level. 


Figure 16 

Flat Windows Located Within the Conncting Node Wall 
(Concept by Bell and Trotti, Inc., 1985) 
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Figure 17 


Flat Windows Located Within a Special Turret 
Attached to a Connecting Node 
(Concept by Bell and Trotti, Inc., 1985) 



Referring to Figure 17, the turret concept affords excellent external visibility with a 
dedicated control station. It also permits flat glass panes to be used of moderately small 
dimensions. 
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Figure 18 


Hemispheric Window Located Within a Special Turret 
Attached to a Connecting Node 
(Concept by Bell and Trotti, Inc., 1985) 



VL FULL WINDOW ASSEMBLY REPLACEMENT 

All manned U.S. space vehicles prior to Space Station have been flown back to earth 
(except Skylab). In most cases (e.g., Mercury, Gemini, Apollo) the capsule was not designed 
for more than one flight so that the windows were analyzed post-flight and then stored. The 
Orbiter windows are inspected after each flight with replacements/repair made as necessary 
on earth. For Space Station, however, it will be necessary to do all inspection, maintenance, 
and replacement on-orbit. This requirement raises some very interesting and challenging 
human factors questions and calls for new and creative structural designs. 

Development of window pane crack development monitoring system should receive 
high priority along with the actual engineering design of the windows themselves. Such a 
monitoring system would significantly reduce the chance that small developing cracks would 
progress to a fracture point. Should an entire window assembly require replacement on- 
orbit, the human factors impact upon the entire crew would (likely) be enormous. One pos- 
sible approach to window assembly replacement is illustrated in Figure 19 (from Bell and 
Trotti, 1985). 
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Figure 19 


Full Window Assembly Replacement Concept 
Proposed by Bell and Trotti (1985) 



Some of the operational procedures which this approach would involve include: (l) all 
consequences of lowering the internal air pressure to zero (e.g., isolating the entire module or 
section of module, donning a pressure suit, etc.), (2) locating, unstowing, transporting new 
window assembly to installation area, (3) removing damaged window assembly, (4) clean- 
ing and seating wall members to receive the new assembly, (5) cleaning, installing, and 
correctly seating the new assembly, (6) packing, mar-king, inventorying and stowing the 
damaged assembly, (7) repressurizing the module (or part of module), (8) performing window 
assembly integrity checks, and (9) preparing, unstowing, stowing, updating, referring to vari- 
ous procedural manuals. 


VII. S UMMAR Y AND CONCL US IONS 

This brief presentation cannot begin to cover all of the myriad human factors associ- 
ated with the design, placement, installation, check-out, monitoring, maintenance, spares 
storage, inventory control, and replacement of PROX-OPS windows. Some of these subjects 
are treated in greater detail elsewhere (Haines, 1986). This paper has outlined some basic 
design philosophy and assumptions for PROX-OPS windows for Space Station. The human 
factors engineer should be brought into the design process as early as possible in order to 
reduce the chance that critical window design characteristics will be incorporated which will 
reduce the operational capabilities of the windows. 
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From Spatial Displays to Spatial Instruments: 
Perceptual Issues in the Design of Perspective Displays 


Stephen R. Ellis 
NASA Ames Research Center 
Moffett Field, Ca. 94035 


I. The Search for the Natural Display 


A. Properties of a natural display 

1. Stimulus response concordance 

2. Appropriate resolution 

3. Minimum perceptua? bias 

B. Spatial displays and spatial instruments 

1. Defn. of spatial display: photo or picture 

2. Defn. Spatial instrument: ruler in photo, clock 

- spatial properties of the display transmit quantitative 
spatial information to user at appropriate degree of 
approximation 

II. Example of a Spatial Instrument: Cockpit Traffic Display 

A. TELERAN 

1. 1940 , s 

2. Old technology 

3. North-up plan-view 

B. Plan-view vs. perspective formats 

1. Better display of vertical separation 

2. Perspective projection, though natural is not enough 

a. Not informative in itself 

b. Potentially misleading 

C. Visual enhancements of perspective displays 

1. Symbolic 

a. realistic models 

b. grid for 3D sense and horizontal metric 

c. reference lines clarify position & aspect 

d. ownship relative metrical aids 

2. Geometric 

a. projection to keep object ini view w/ varying fov 

b. scaling of aircraft independent of position for visibility 
b. differential vertical/horizontal scale 

D. Pilots Behavior Effected by Difference b / Plan & Perspective 

E. Generic perceptual issues 

III. Definition of a Perspective Display 

A. Perspective parameters 
1. 3D geometry 

a. 2 coordinate system: world and eye 

b. center of projection, viewing vector 

c. Azimuth, elevation, view plane or picture plane 
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2. 2D geometry 

a. D - distance to reference point 

b. d - distance to picture plane 

c. FOV 

3. Samples of effects of parameter changes 

a. height of COP: effect on depicted direction 

b. FOV changed alone: effect on size 

c. Compensated FOV and D to keep part ol picture constant: 
effect on distortion 

B. Visual enhancements 

1. Use of parameters to improve accuracy of spatial sense _ 

2. Addition of metrics and symbols 

IV. Investigations of Perceptual and Motor Effects 

Quantitative and objective development of geometric and symbolic 
enhancements 

A. Direction judgements 

1. The direction judgement task 

2. Examples of different FOV conditions w/ compensated size 

3. Distortion angle geometry: change FOV * eye not at COP 

4. Raw azimuth data showing FOV effect and fit 

5. Basic pattern of relative azimuth error 

B. SD Cursor control 

1. Alternative measure of spatial sense 

2. Examin effect of azimuth and elevation of viewing vector 

3. Defn: of SD tracking and example displays 

4. Coarse look at 3D tracking: monoscopic & stereo 

- defn: normalized RMS error 

- stereo better by enhancement equates mono 

- reference lines seem key for this task 

5. Finer look at perspective parameters 

- FOV alone 

- FOV &; compenstated distance D: distortion angle 

- Elevation angle: 45 deg optimum 

- Azimuth defn: and coarse look 

- Azimuth Finer look at 1 subject: learning, optimum at 10 deg 

- Azimuth joystick rotation, relavence to camera angles 

V. Future Directions: Window on the Synthetic Universe 

A. Other perceptual and control tasks & effects 

1. Pick and place robotics: = to 3D tracking, uncanny 

2. Visual acquisitiion: egocentric pointing 

study of elevation parallel to azimuth 

B. Research questions for proximity operations displays 

1. Intuitive spatial displays for nonintuitive orbital dynamics 

2. Use of spatial displays in microgravity 

- affect on exocentric orientation? 

- egocentric orientation 

3. Three-dimensional target designation 

4. Proxop specific enhancements 

- local axes for communication: MMU example 
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C. Perspective display design recommendations 

1. Field of view angle 

2, Viewing vector direction and control axes 

S. Scaling and center of projection 

4. Metrical aids 

5. Resolution of aspect ambiguity 
D. Augmenting the Intelligent Eye 

1. Gregories worries 

2. Bekesy’a equipment as an example 

3. Role of visual enhancements in the synthetic universe 
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IMAGE MANAGEMENT RESEARCH 


Andrew B. Watson 

Perception & Cognition Group, NASA Ames Research Center 

Space Station Human Factors Research Review 
December 3-6, 1985 

What is an Image Management System ? 

To begin with, what is an image ? I will take as a working definition any picture 
originally captured from life, as opposed to pictures generated digitally, which I will 
call graphics . Thus images are typically the result of a photographic process, either 
conventional or digital. Another distinction is that images are typically pixel 
oriented , while graphics are object oriented (eg lines, circles, areas etc). Now what is 
an image management system? Computers today provide an extensive set of tools 
for manipulating text, and a somewhat smaller and newer set for dealing with 
graphics. The next revolution will be the provision of tools for manipulating digital 
images. An image management system is a computer-based facility for capturing, 
coding, processing, editing, storing, analysing and displaying images. 

Space Station Applications 

What is the need for such systems on board the space station and in the various 
space station supporting centers? The current Space Station Flight Operations 
Requirements document, dated November 1985, is replete with references to the 
need for video interfaces among the station, platform, shuttle, omv's, and various 
ground control centers. (2.1.7.1, 2.1.9.1, 2.1.10.1, 2.2.3.1, 2.2.4.8, 2.2.6.6, 2.2.12.4, 3.1.5.1). 
While perhaps originally concieved as analog video hook-ups, it seems highly likely 
that the advantages of digital video will eventually lead to its adoption, particularly 
in during EOC. 
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Elsewhere it is stated that these video links must be part of a highly reconfigurable 
"General Purpose workstation" (2.2.4.7, 2.2.4.8), which can only be done effectively 
via digital video. This integration of digital video into general workstations is itself 
an image management problem. 

The requirements document also deals at some length with training, noting the 
need for video aids (2.1.11.3). This presumably refers to video disc, with its 
invaluable capacity for random access. It is further stated that comparable training 
must be provided on board the space station (2.2.9.1). We are thus lead to assume 
that the space station will include onboard storage for large random access libraries 
of training images. 

Elsewhere it is stated that "Automated training shall utilize operational onboard 
equipment in a simulation mode." (2.2.9.4) Since most operations will require 
visual/video monitoring, this would seem to require an onboard library of imagery 
to accompany the simulation. 

Other space station image databases can be imagined, such as parts directories, 
archives of scientific imagery, repair manuals, personnel directories, and so on. 

Research Issues 

What are the research issues involved in image management systems? They divide 
into two sorts: image processing research and image perception research . The image 
processing issues are the traditional ones of digitizing, coding, compressing, storing, 
analysing, and displaying, but with a new emphasis on the constraints imposed by 
the human perceiver. For example, the efficiency of a coding scheme is considered 
not with respect to statistical efficiency alone but also with respect to perceived 
fidelity. We have made some progress in this area and have developed two image 
coding algorithms that may greatly increase the efficiency of an IMS. 
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The second category, image perception research, involves a study of the theoretical 
and practical aspects of visual perception of electronically displayed images. We are 
interested in issues such as how rapidly a user can search through a library of 
images, and how to make this sort of search most efficient? Is it best to present each 
image at full size and full resolution, or is it better to present many images at once at 
reduced size and resolution? This raises a fundamental question, namely, what are 
the effects of size and resolution upon the speed and accuracy with which humans 
recognize images. We have experiments on this question underway at the moment. 

Another set of issues relate to the optimal interface to an IMS. What image 
manipulations should the user be able to do? Zooming? Moving images about the 
screen? Accessing aditional textual data about the image? Subtracting two images? 
More general image processing operations? Some of these are no doubt useful in 
particular applications but is there a set which are highly useful in the "generic" 
setting? 

Another large and fascinating issue is how to code images in a way that is optimal 
for the human perceiver. Recent research in our group suggests that an 
understanding of human spatial and color perception may provide ways to 
massively reduce data requirements without sacrificing visual fidelity. To give but 
one example, it is well known that color vision has lower spatial resolution than 
pattern vision, and an image coding method which could take adavantage of this 
would result in a massive data compression. One algorithm that we have 
developed, which separates an image into separate bands of resolution, provides a 
natural way of doing this. 
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IMS Test-Bed 


We have designed a test-bed within which many image management issues can be 
addressed. It consists of a high performance UNIX workstation with a very high 
resolution color framebuffer and display. The workstation hosts a digitizer, an array 
processor, and an optical video disc recorder. 

The typical scenario is that the user, comunicating through an interface resident on 
the workstation, requests an image or set of images. The locations of these images on 
the optical disc are determined by a database resident on the workstation. The images 
are played back and captured by a digitizer. The digitized images are processed by the 
array processor and displayed with desired size, location, and timing on the high 
resolution framebuffer. Images can be recorded on the optical disc (and entered into 
the database) either from a video camera or from the digitizer. This configuration 
sacrifices something in speed to obtain the high degree of flexibility suitable to a 
research, rather than production, environment. It is currently about 75% complete. 
When complete, it will have the following capabilities: 

Capabilities of Test-Bed 

rapid aquisition from large (24,000) image database 

rapid digital processing of selected images 

display of multiple images of arbitrary size on single screen 

arbitrary presentation rate of full-size images 

database management of image library 

supervisory control of all functions from UNIX workstation 
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Conclusion 


There is no question that image management systems will become an integral part 
of the information systems of the near future, and it is almost as certain that they 
will become part of the space station. We hope that our research will help make that 
system powerful and useful, and generally contribute to the efficiency of space 
station operations. 
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IMAGE MANAGEMENT RESEARCH TEST-BED 


VME 
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NASA-AMES WORKLOAD RESEARCH PROGRAM 


Sandra Hart 

NASA Ames Research Center 


During the next hour, I will describe the purpose, philosophy, 
structure, and some of the accomplishments of the Human Performance 
Research Group of the Aerospace Human Factors Research Division. I will try 
to demonstrate the flow of information from generic, theoretical research to 
specific space- s tat ion related applications. 

Although an increasing emphasis has been placed on providing computer- 
based automation in every phase of modern systems, the decision has been 
made that man will continue to play a central role in space station 
operations. Humans have capabilities beyond those of the most sophisticated 
computer systems and their flexibility and adaptibility provides a unique 
asset in such a remote environment. The activities that will be performed 
in the Space Station range from direct control of spacecraft (e.g., the 
orbiter, the orbital transfer vehicle, and the manned maneuvering unit) to 
indirect control (e.g., the orbital maneuvering vehicle and the remote 
manipulator arm), to housekeeping activities and the conduct of scientific 
experiments. Each will require specialized training, take a certain amount 
of very limited and precious time and will have some associated human (e.g., 
workload) and payload cost. 

The space station provides a unique situation in which teams of 
astronauts, scientists, and technicians will live and work in an unfamiliar 
environment for prolonged periods of time. Space flight has traditionally 
required high levels of performance in relatively stressful environments. 
The stressors may include isolation from familiar work and living 
surrounding, physiological discomfort associated with weightlessness, and 
potentially high levels of workload. Major changes in the U. S. Space 
Program may precipitate additional problems, such as longer missions, 
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hetereogeneous crews, more varied and complex tasks, and an expected 
decrease in the training provided for individual crewmembers. The 
increased emphasis on space commercialization will require crewmembers to 
exhibit new levels of productivity. 

Even though previous space missions have proven to be extremely 
successful, the available evidence suggests that the performance and 
reliability of the human elements of aerospace systems is curently lower 
than that of other elements. Studies of human reliability show that most 
human-related errors involve inadequate or faulty crew coordination and 
inadequate or faulty man-machine interface. These problems are soluble. 
One of the goals of our program is to evaluate ways to predict the impact of 
performing a large range of tasks on the human operator and to provide 
guidelines for design and operation to enhance system performance and 
optimize human behavior and experience. 

It is important to assign humans those tasks with which they can excell 
and to redesign, aide, automate, or eliminate those tasks which they perform 
poorly, unreliably, or with unacceptably high levels of workload. In 
addition, the presentation of information and control inputs must be 
designed so as to optimize human capabilities. In order to accomplish 
this, predictors and measures of human performance and workload are needed 
to evaluate the effectiveness of display, control, and automation options so 
as to maximize the efficiency, effectiveness and reliability of the human 
element in a man-machine system. This information is required early in the 
design and construction process, as retrofits and modifications are costly 
and time-consuming, if not impossible, once the actual construction process 
of the space station has begun. 


Traditional measures of human performance (which focus on lower level, 
in-the-loop control) may not be applicable for high-level supervisory 
control tasks nor the measurement of productivity, efficiency, information 
seeking, decision making or control strategy for teams of operators. In 
addition, the impact of crewmembers' efforts to accomplish mission 
requirements on the human operators themselves (e.g., workload) is an 
important design consideration. 
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OUTLINE: 

O ORGANIZATION OF PROGRAM 

- PROBLEM/OBJECTIVES/APPROACH 

- RESOURCES 

- COLLABORATION 

- CONCEPTUAL FRAMEWORK 

0 CRITERION TASKS 
0 PREDICTIVE MODEL 

0 ASSESSMENT TECHNIQUES 

- PERFORMANCE 

- PHYSIOLOGICAL 

- SUBJECTIVE 

0 VALIDATION/APPLICATION OF TECHNIQUES 


Research has been underway at Ames for several years to develop valid 
and reliable measures and predictors of workload as a function of operator 
state, task requirements, and system resources. Although the initial focus 
of this research was on aeronautics, the underlying principles and 
methodologies are equally applicable to space, and provide a set of tools 
that NASA and its contractors can use to evaluate design alternatives from 
the perspective of the astronauts. I will begin by describing the 
objectives and approach of the research program, the resources used in 
conducting research, and the conceptual framework around which the program 
evolved. Next, I will describe the standardized tasks, predictive models 
and assessment techniques we have developed, and their application to the 
space program. Finally, I will review some of the operational applications 
of these tasks and measures. 
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PROBLEM: 


0 NONOPTIMAL LEVELS OF WORKLOAD IMPOSED ON THE HUMAN OPERATORS OF ADVANCED 
SYSTEMS ARE A SIGNIFICANT FACTOR IN THE EFFICIENCY AND SAFETY OF SYSTEM 
OPERATIONS. OVERALL SYSTEM PERFORMANCE, TRAINING REQUIREMENTS. ADDITIONAL 
HARDWARE AND SOFTWARE COSTS. CREW COMPLEMENT. AND JOB SATISFACTION. 


o SINCE WORKLOAD REFLECTS THE INTERSECTION BETWEEN A PARTICULAR OPERATOR 
PERFORMING A SPECIFIC MISSION. USING THE AVAILABLE HARDWARE. SOFTWARE AND 
HUMAN RESOURCES. WORKLOAD HAS MULTIPLE CAUSES AND EFFECTS. 

0 THUS. DIFFERENT WORKLOAD QUESTIONS AND CIRCUMSTANCES REQUIRE DIFFERENT 
MEASURUREMENT TECHNIQUES. 

0 STANDARDIZED. VALIDATED. AND SENSITIVE MEASURES ARE NOT YET AVAILABLE TO 
EVALUATE THE WORKLOAD OF EXISTING SYSTEMS NOR TO PREDICT THE WORKLOAD 
OF PROPOSED SYSTEMS DURING THE DEVELOPMENT PROCESS. 


A resurgence of interest in the field of workload assessment was 
prompted by the Presidents Task Force on Crew Complement, It became clear 
that the question of whether or not two or three crewmembers would be 
required for the next generation of aircraft could not be answered 
satisfactorially without a clear concept of what factors affected crew 
workload, how workload could be measured, how much workload is too much (or 
too little), the relationship between measures of workload and performance, 
and the effectiveness of automation in reducing or redistributing workload. 

Our initial premise was that nonoptimal levels of workload are a 
significant factor in efficient and safe system operations, training 
requirements, required hardware and software, crew complement, and job 
satisfaction. Since workload reflects the intersection between a particular 
operator performing a particular mission, using the available hardware, 
software and human resources, workload may have multiple causes and effects. 
Thus, different workload-related questions and circumstances require 
different measurement techniques. Even more important, for practical 
reasons, is the need for standard, valid, sensitive techniques to predict 
the workload of proposed systems early in the design process. 
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"COST " OF FULFILLING MISSION REQUIREMENTS 


SYSTEM RESOURCES 
HARDWARE 
SOFTWARE 



STRESS 

FATIGUE 

DISSATISFACTION 


PERFORMANCE 

DURATION 

PRECISION 

SAFETY 

RESERVE 


The "cost" of fulfilling mission requirements can be conceptualized in 
many ways. It can be quantified in terms of system resources required; the 
amount and sophistication of hardware and software required and the number 
and qualifications of personnel. The cost of the training required for 
crewmembers to accomplish mission objectives using existing equipment can be 
quantified as well, as can the cost of failure to meet mission objectives. 
We define the "cost" to human operators of performing their part in a man- 
machine system as workload. Workload is more difficult to quantify in 
objective terms than the other costs of system performance. It's impact may 
be evalutaed indirectly, however, through lowered levels of performance, 
additional required resources or training, and operator dissatisfaction. In 
order to meet mission requirements, there may be a tradeoff between 
additional resources, additional training or higher levels of workload. If 
operators are already working at their peak efficiency, then lower levels of 
performance might have to be accepted or additional system resources 
provided. 
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PROGRAM OBJECTIVE: 

DEVELOP AND VALIDATE TECHNIQUES TO PREDICT AND ASSESS THE EFFECTS 
OF TASK DEMANDS. ENVIRONMENT. AND TRAINING ON OPERATOR BEHAVIOR 
WORKLOAD. AND PERFORMANCE. 




APPROACH: ... . :■/ 

' \ - 3'. ; : X > V 

PERFORM GENERIC RESEARCH TO DISCOVER UNDERLYING PRINCIPLES. DEVELOR|§li|| 

AND VALIDATE ASSESSMENT TECHNIQUES. AND CREATE PREDICTIVE MODELS 

, . . - \ ■ , ■ • • ■ , 

PERFORM VEHICLE-SPECIFIC APPLICATIONS OF GENERIC CONCEPTS AND METHODS-I^ 
TO ADDRESS OPERATIONAL PROBLEMS. - P \ 


Our asumption is that workload is a hypothetical construct that 
represents the cost to human operators of achieving mission objectives. 
Thus, our definition is human-centered, rather than task-centered. An 
operator 1 s experienced workload representes many other factors in addition 
to the objective demands placed on them. It is not an inherent property of 
a task but emerges from the interaction between the requirements of the 
task, the skills and behaviors of an operator, and the circumstances under 
which the task is performed. 

The initial goal of the program was to develop measures and predictors 
of human workload that took into account all of the relevant factors. 
Several parallel lines of research were undertaken in which underlying 
principles were discovered, measurement techniques developed and validated, 
and predictive models created. Vehicle-specific applications of these 
generic concepts and methods were performed concurrently to address a 
variety of operational problems. 
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The initial focus of the research was on assessment. The focus moved 
toward predition as the theoretical problems associated with assessing 
workload in existing systems were resolved. I will describe the results of 
this research in greater detail in a moment. More recently, our focus has 
been on training. Specifically, we wish to investigate the 
interrelationships among workload, training, and performance in highly 
automated systems, such as the LHX helicopter and the space station. 

The focal point of this area of research is a workshop sponsored by 
NASA that will be held in January. The workshop participants will consider 
how to quantify and predict performance and workload changes as training 
progresses, and, conversely, to determine the role of workload in training 
effectiveness. The proceedings of this workshop will be published in a book 
for public dissemination. The specific focus of the discussions will be on 
the two vehicles that represent two workload and environmental extremes 
faced by technology - - single-pilot, nap-of-the earth helicopter flight at 
night during the performance of Army missions and Space Station operations. 
Training may well emerge as a significant problem area in space station 
operations. Due to new mission goals and characteristics, it is anticipated 
that the training time allowed for space station operators will be reduced. 
Some of the training now accomplished on the ground may be performed in 
orbit and recurrent training may be required on orbit due to the extended 
mission durations. More effective and efficient training programs, 
particularly those that focus on understanding and operating highly 
automated subsystems, will be needed to maintain workload and performance at 
acceptable levels. 
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RESEARCH GRANTS FUNDED BY THE PROGRAM 


VIRGNINA POLYTECHNIC INSTITUTE 

ARIZONA STATE UNIVERSITY 
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UNIVERSITY OF TORONTO 
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UNIVERSITY OF SOUTHERN CALIFORNIA 
WAYNE STATE UNIVERSITY 
STANFORD UNIVERSITY 
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LYMAN 
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JORDAN 
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SWINEY 

SWEENY 

WICKENS, KRAMER 

KANTOWITZ 

MORAY 

GOPHER 

KANTOWITZ. TOWNSEND 

HANCOCK 

FRANKEL 

CALFEE 




RESEARCH CONTRACTS FUNDED BY THE PROGRAM 


GENERAL PHYSICS CORPORATION 
STRUCTURAL SEMANTICS 
STANFORD RESEARCH INSTITUTE 
DOUGLAS AIRCRAFT COMPANY 
SEARCH TECHNOLOGY 


GOMER 

LINDE. GOGUEN 
CHESNEY 
BIFERNO 
ROUSE 


Our program represents an active collaboration between inhouse 
research, joint research with other government agencies and industry, and 
research funded through grants and contracts. The personnel involved in the 
program include psychologists, pilots, and engineers. The facilities used 
range from laboratory settings to part-task simulations, full-mission 
simulations, and inflight experiments. The research efforts differ with 
respect to theoretical perspective, assessment techniques used, research 
facilities, and focus (theoretical or applied, prediction or assessment). 
For each critical area, several different lines of research have been 
undertaken. Coodination and integration has been accomplished though 
publications and scientific presentations, meetings, and shared experimental 
tasks and measurement techniques. 
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INTERACTIONS WITH OTHER AGENCIES: COLLABORATIVE RESEARCH 


X ARMY (COEC) 

* ARMY (AVSCOM) 


* NASA-JSC 


X FAA 


X Navy (NATC) 


SCOUT II Helicopter Experiment 

COBRA/Pilot Night Vision System Inflight Training 
1 vs 2 Pilot (AOOCS Simulation in VMS) 

ARTI Contractor Simulations-Government scenario 

Space Suit Comparison 

RMS Workload Prediction/Evaluation 

TCAS Workoad Evaluation (MVSRF B-727 simulator) 

Tilt-rotor Workload Evaluation 


x Air Force (Brooks) Pilot Recertification Test Battery 


x British CAA 


North-Sea Oil Operations Workoad Evaluation 


We have played a support role in a number of simulation and inflight 
experiments conducted by outside organizations. In general, we provided 
workload assessment methodologies and application procedures to assist these 
organizations in addressing oeprationaly relevant workload-related problems. 
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MANPMNT- 

I MANPOWER & PERSONNEL 
INTEGRATION 



• HUMAN FACTORS 
ENGINEERING 

• MANPOWER 

• PERSONNEL 

• TRAINING 

• SYSTEMS 
SAFETY 

• HEALTH HAZARD 
ASSESSMENT 



NASA CONTRIBUTION: 

o BRIEFING: OVERVIEW OF WORKLOAD AND PERFORMANCE ASSESSMENT RESEARCH 

0 COURSE SYLLABUS: BASED ON NASA WORKLOAD REPORT 



0 COMPUTER-BASED TRAINING PROGRAM: NASA “EXPERT" SYSTEM FOR 

SELECTING WORKLOAD ASSESSMENT METHODOLOGY 


IS 

m 

| 

$ 



Operational validity and applicability have been insured by frequent 
involvement in addressing operational problems posed by members of other 
organizations- One example of such involvement is the role that we played 
in the development of the Army MANPRINT course. This program represents a 
major effort by the Army to integrate human factors issues, manpower and 
personnel, and training into the materiel acquisition process. The results 
of our research provided the foundation for the course presented by the Army 
to familiarize Army managers with human factors engineering and several of 
the programs developed at Ames will be used as teaching aides. 
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COLLABORATIVE RESEARCH: 

ADVANCED DIGITAL OPTICAL CONTROL 

SIMULATION (ADOCS) 

OBJECTIVE: 

(1) COMPARE ONE vs TWO PILOT WORKLOAD 

(2) COMPARE WORKLOAD OF DIFFERENT 
COMBAT MISSIONS 
EVALUATE WORKLOAD IMPACT OF 
DIFFERENT LEVELS OF AUTOMATION 



One example of such joint research is a recent 
completed with the Army Aerof lightdynamics Division, 
was to compare the workload of pilots flying 
configurations with different levels of automation, 
missions that an LHX-type helicopter might perform 
flights were performed in the Ames Vertical Motion 
Advanced Digital Optical Control Simulation (ADOCS). 


simulation which we 
The goal of this study 
one- or two-pilot 
The tasks represented 
in the 1990s. The 
Simulator using the 
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IMPOSED 

WORKLOAD 


OPERATOR BEHAVIOR 


SUBJECTIVE 

EXPERIENCE 


JCEPTION! 

I 


.PERFORMANCE 


TASK VARIABLES 

OBJECTIVES: GOALS 

CRITERIA 

TEMPORAL 

STRUCTURE: DURATION 

RATE 

PROCEDURES 

SYSTEM 


SELECTION OF STRATEGIES 
OPERATOR CAPABILITIES 
SENSORY/MOTOR SKILLS 
COGNITIVE SKILLS 
KNOWLEDGE BASE 
COMMITMENT OF RESOURCES 
PHYSICAL 
MENTAL 


SPEED 

ACCURACY/PRECISION 

RELIABILITY 

RESOURCES: INFORMATION 

EQUIPMENT 
PERSONNEL 

OPERATOR QUALIFICATIONS 

( 




ENVIRONMENT: SOCIAL 

PHYSICAL 

V 


1 


INCIDENTAL VARIABLES 
SYSTEM FAILURES 

> 



CONSEQUENCES OF 
PERFORMANCE 

OPERATOR ERRORS 
ENVIRONMENTAL CHANGES 
STATE OF THE OPERATOR 


OPERATOR'S PERCEPTION OF : 


DIRECT FEEDBACK 
KNOWLEDGE OF RESULTS 


- TASK GOALS & STRUCTURE 




PERFORMANCE 
PRECONCEPTIONS & BIASES 





PHYSIOLOGICAL 

CONSEQUENCES 


As I mentioned before, the focal point of the program was a conceptual 
model in which task-related, behavior-related, and operator-related 
variables were related to each other. Imposed workload refers to the 
situation encountered by a specific operator or team of operators in 
performing a task. The intended demands of a task are created by its 
objectives and performance criteria, temporal structure, system resources 
provided and the environment in which it is performed. 

Task objectives are particularly critical because they determine the 
target performance levels that operators attempt to achieve. The temporal 
structure of the task refers to the length of time available to perform the 
task or subtask elements, the degree to which task elements overlap in time, 
the procedures and organization, and the degree to which operators can 
select which tasks to perform and in which order. The objectives and 
temporal structure of a task create the task requirements. This can be 
distinguished from the workload associated with the system resources 
provided to the operators to perform such a task. 

System resources refer to the information, equipment, controls, 
displays, and personnel that are provided to assist the operator in 
performing the task. System resources include automation that has become 
such an important element in most advanced systems. A major focus of our 
research program has been to investigate the workload- impact of different 
types of automation on operator workload. In general, the trend has been to 
reduce the physical workload of operators and to remove them from in-the- 
loop control activities, but often at the cost of an increase in mental 
workload. An additional concommitant of automation has been to alter the 
nature and impact of operator errors - - relatively "minor" typographical 
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errors can lead to extremely grave consequences that are difficult to detect 
becasue the operator is not sufficiently integrated into the performance of 
the system. 

The environment can have a significant effect on operator workload and 
performance. The social environment, that is crew interactions, leadership 
styles, group dynamics, can all play a significant in the safe and efficient 
functioning of a crew. This particular issue will become particularly 
salient in space station operations, where crew members live and work 
together in a very confined environment for a prolonged period of time. The 
physical environment refers to the workstation layout, personal space, 
climate, threat from man-made or natural sources. 

Each time a particular task is performed by a specific operator, 
incidental variables may occur that can alter the workload demands of the 
task either subtly or substantially. In this regard, the primary focus of 
our research efforts has been to examine the role of system failures and 
operator errors on subsequent task performance and crew workload. We 
consider errors to be a potent source of workload rather than an indicator 
of workload. The disruption caused by errors is particularly acute for 
well- trained operators, as they must step out of over- learned , automatic 
patterns of behavior to diagnose and solve the error and then continue with 
the interrupted activities with conscious attention. 

System response refers to the behavior and accomplishments of a man- 
machine system. Operators are motivated and guided by the imposed demands, 
but the strategies selected and effort exerted reflects the operators 
perception of what it required of them. In most tasks, a variety of 

strategies are possible and different tasks, obviously, required different 
skills and capabilities. Thus, the role of human behavior in workload can 
be complex. Physical effort is the easiest to conceptualize and measure, 
but its contribution to advanced systems in diminishing. The problems 
associated with physical effort exerted in zero-G environments should be 
relatively unique, as the astronauts cannot rely on highly overlearned (and 
thus automatic) patterns of motor behaviors learned in a one-G environment. 
This source of workload - - that is the conscious attention to physical 
activities that are normally performed without conscious attention should be 
relatively great early in a mission, but should be reduced as time on orbit 
increases, and new patterns of response are developed. Mental effort serves 
as a potent intervening variable between measurable stimuli and measurable 
responses but it is difficult to quantify directly. It is unlikely that 
this aspect of human workload should be affected significantly by a zero-G 
environment, except for those aspects involved with motor control and 
spatial orientation. 

Performance represents the product of the operators* actions and the 
limitations, capabilities and characteristics of the system controlled. 
Performance feedback provides information to the operators about their 
success in meeting task requirements, the appropriatness of the strategies 
selected, and the level of effort exerted, allowing them to modify their 
behavior to achieve more acceptable levels. We have examined performance 
from two perspectives: (1) As an indicator of the degree to which operators 
were able to satisfy task requirements and (2) As an indicator of the cost 
incurred by the operator in doing so. Performance levels tend to remain 
fairly constant as long as the task requirements remain within the 
oeprator*s capabilities. In this case, performance measures do not reflect 

41 



the increasing levels of effort associated with meeting progressively 
increasing task demands. When performance requirements exceed operators 1 
capabilities, or they lower their performance standards, decreasing levels 
of performance may in fact reflect the existence of higher levels of 
workload . 

The consequences of performing a task on an operator can be 
physiological or subjective. Since operators may not be aware of every task 
variable, the processes that underly their decisions and actions, or the 
influence of preconceptions about the task, workload experiences may not 
reflect all of the relevant factors and may, in fact, reflect some that are 
irrelevant. Thus, we draw a distinction between the level of workload that 
a system designer intended to impose on an operator, the responses of a 
specific man-machine system to the task, and the operators' subjective 
experiences. The importance of subjective experiences extends beyond their 
association with subjective ratings, however. The phenomenological 
experiences of human operators affects subsequent behavior, and thus, 
performance. If operators consider the workload of a task to be excessive, 
they may adopt strategies that are appropriate for high workload situations 
(such as shedding tasks, hurrying, or accepting lower levels of 
performance) and they may experience psyiological or psychological distress. 

One example of a misperception of task requirements was presented to us 
by JSC as a problem requiring an experimental solution. The mission 
commander on an early Shuttle flight reported experiencing "time 
compression" during approach and landing - - that is the feeling that time 
was passing too quickly. One suggestion was that experiencing zero-G had 
somehow disrupted his ability to perceive the passage of time accurately. 
The more likely explanation, based on a series of experiments, was that 
failures of time perception is a common concommitant of stress and high 
levels of workload. 

Physiological responses may reflect momentary responses to task 
demands (such an elevated heart rate or pupil dilation) or relatively long 
term effects following prolonged exposures. It might be expected that this 
aspect of operator's responses to workload might be relatively more extreme 
in orbit, as task-related stressors might interact with environmental 
stressors associated with zero-G. 
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CRITERION TASKS DEVELOPED 
AT AMES: 

0 FITTSBERG 

0 POPCORN 

0 MULTI-COCKPIT 
SIMULATION 

0 STANDARD FLIGHT 
SCENARIO MODEL 



WORKLOAD CAN NEVER BE MEASURED ABSOLUTELY 
(WHAT WOULD THE UNITS BE?) 

THflT HflVE BEEN CAL I BRATEo| 
AN APPROPRIATE CRITERION TASK(S) 

CAN HAVE A COMMON RELATIVE REFERENCE POINT 


The fact that workload validation procedures are often circular 
presents a significant problem in the development and validation of 
candidate workload measures. since there is no objective standard against 
which a measure can be compared, the decision of whether or not it is 
sensitive is often made ad hoc. That is, if the measure varied in 
accordance with the supposed levels of workload imposed by the task, the 
assumption is that it is sensitive, and if it does not, it may either 
indicate that the measure was not sensitive or that the experimenter did 
not, impose the intended levels of workload. 

For this reason, we have developed a set of "criterion tasks", for 
which standardized levels of workload can be created according to well - 
known psychological principles. These tasks represent stylized versions of 
the activities that operators normally perform in advanced systems. 
Candidate measures or models can then be compared against known workload 
levels imposed by these tasks. I will describe two such tasks. 
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CRITERION TASKS: FITTSBERG 


OBJECTIVE: 

DESIGN A SIMPLE. RELIABLE. AND FLEXIBLE LABORATORY TASK IN WHICH 
TASK ELEMENTS ARE FUNCTIONALLY RELATED BUT: 

(1) RESPONSE SELECTION AND RESPONSE EXECUTION DIFFICULTY CAN BE 
MANIPULATED INDEPENDENTLY 

(2) PERFORMANCE ON SUBTASK ELEMENTS CAN BE MEASURED INDEPENDENTLY 
APPLICATIONS: 

(1) IDENTIFICATION OF SUBTASK ELEMENTS TO AUTOMATE 

(2) DISPLAY MODALITY (AUDITORY/VISUAL) 

(3) DISPLAY FORMAT (SPATIAL/VERBAL/NUMERIC) 

(4) PREDICTION OF COMPLEX TASK PERFORMANCE 

(5) SUBJECTIVE ASSESSMENT OF NON-HOMOGENEOUS INTERVALS 

(6) IMMEDIATE vs RETROSPECTIVE WORKLOAD EVALUATION 

(7) ASSOCIATION AMONG MEASURES OF WORKLOAD AND PERFORMANCE 
(B) BASIS OF SPACE SUIT EVALUATION TEST BATTERY 

(9) PRIMARY TASK FOR CURSOR CONTROL EVALUATIN IN SHUTTLE 


The "Fittsberg task" is a simple, flexible laboratory task where 
subtask workload levels can be independently manipulated and measured over a 
wide range. It provides an alternative to the traditional dual task 
paradigm in which two unrelated tasks are performed during the same time 
interval. It represents the types of tasks that are performed in many 
automated systems: a requirement for action is recognized and the 
appropriate plan of action selected. The plan of action is executed by an 
automated system in response to a discrete command. 
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"FITTSBERG" TASK 

A TARGET ACQUISITION TASK (DIFFICULTY INDEXED BY FITTS LAW) 



Fittsberg task components are functionally related - response 
selection provides information for and initiates response execution. The 
response selection task is a target acquisition based on Fitts* Law. Two 
identical targets are displayed equidistant from a centered probe. The 
decision about which target to acquire is based on a Sternberg memory search 
task; Subjects acquire the target on the right if the information presented 
in the center of the display is the same as a remembered value or the target 
on the left if it is not. A wide variety of response selection tasks have 
been used in addition to the Sternberg Task - - mental arithmenic, pattern 
match, time estimation, etc. Workload levels of one or both task components 
can be held constant or systematically varied within a block of trials. The 
stimulus modality of the two components can be the same (visual/visual) or 
different (auditory/visual) . 

Response selection performance is measured by RT and percent correct. 
Response execution performance is measured by MT. RT, but not MT, increases 
as the difficulty of the response selection task is increased. MT, but not 
RT, increases as target acquisition difficulty is increased. Workload 
ratings for the Fittsberg task integrate the influences of the component 
subtask components. Workload ratings and performance levels for the 
combined task are often substantially less that would be predicted by simply 
adding single-task workload ratings or response times . 
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WORKLOAD MEASUREMENT 
FOR SPACE SUIT DESIGN 




PfcRFGHMANCK O. 
*F SEFHCH CKOUP 


OBJECTIVE! 

TO COMPARE ALTERNATIVE SPACE SUIT DESIGNS 
FOR UPPER BODY MOBILITY AND COMFORT 

APPROACH! 

PERFORM TASKS THAT IMPOSE PREDICTABLE 
DECISION-MAKING AND RESPONSE EXECUTION 
WORKLOAD LEVELS BEFORE AND AFTER EXERCISE 

EXPERIMENTAL TASKS: 

EXERCISES: 

TORQUE WRENCH 
BICYCLE EGOMETER 
WEIGHT TRANSFER 
ROPE PULL 
DECISION TASKS: 

RICHT/LEFT CHOICE 
SHORT-TERM MEMORY 
MENTAL ARITHMETIC 
RESPONSE EXECUTION: 

CONTROL STICK DEFLECTION 
TARGET DIFFICULTY 

MEASURES: 

PHYSIOLOGICAL: 

HEART RATE 
OXYGEN UPTAKE 
SUBJECTIVE OPINION! 

COMFORT SCALE 

MULTI-DIMENSIONAL HORKLOAO RATINGS 
PERFORMANCE: 

TASKS COMPLETED % CORRECT 

REACTION TIME MOVEMENT TIME 


This task has proven to be a useful focal point for several space- 
related applications. In response to a request by Johnson Space Center, we 
provided the hardware and software to use the Fittsberg task in a series of 
experiments in which two alternative space suit configurations were compared 
with respect to upper body mobility and comfort. Several Fittsberg tasks 
are performed using either fine or gross arm movements before and after a 
battery of physical exercises are completed. Physiological, subjective and 
performance measures are obtained to aide in the comparison between the two 
suit configurations . 

Again the advantage of using this task is the fact that it has been 
calibrated in advance of the experiment with respect to expected workload 
and performance levels. 
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NASA-AMES WORKLOAD AND PERFORMANCE RESEARCH 

STUDY OF CURSOR CONTROL DEVICES IN ZERO-G 



OBJECTIVE 

• EVALUATE 3 CURSOR CONTROL DEVICES EARLY 
AND LATE IN ZERO-G EXPOSURE DURING FY86 
SHUTTLE MISSION 

APPROACH 

• ARC/UNIVERSITY COLLABORATION 

• ARC-DEVELOPED "FITTSBERG" TASK AS 
CRITERION TASK 

• COMPARISON OF VERTICAL, HORIZONTAL, AND 
ANGULAR MOVEMENTS TO ACQUIRE TARGETS 
WITH: 

- TRACK BALL 

- JOYSTICK 

- ARROW KEYS 


The Fittsberg task was selected for an experiment that will be flown in 
the Shuttle in the fall of 1986. The purpose of the experiment, which will 
be conducted jointly with MIT and JSC, is to evaluate three alternative 
cursor control devices in zero-G. 
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The experimental task will be presented on a Compass-Grid 
microprocessor mounted on an adjustable work surface attached to a Spacelab 
hand rail. Both foot and arm restraints will be provided. The three space- 
rated input devices devices - - track ball, arrow keys, and joystick will 
be positioned with Velcro strips. 
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DISPLAY CONFIGURATIONS FOR CURSOR 

CONTROL EXPERIMENT 

Memory Set « A 



CARDINAL CONFIGURATION 

DIAGONAL 

CONFIGURATION 

Example: 'Easy* Target 

Example: 

’Hard' Target 
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EXPEHIMENTAL DESIGN: 

Each block of 8 trials will be repeated three times (early, 
middle, and late in the mission) by four crewmembers. 


mm 


Cardinal Movements 


Diagonal Movements 


[Keyboard 

rackball 

bystick 


MS= 1 




MS-1 I MS =4 


MS* l MS=4 


MS-1 MS-4 


Twenty-four blocks of Fitsberg trials will be performed during three, 
30-min intervals early, middle, and late in the 7-day mission by four 
mission specialists. The difficulty of the response selection task will be 
manipulated by varying the number of items to be remembered (the Sternberg 
paradigm). The difficulty of the response execution portion of the task 
will be varied by manipulating the direction of movement - - either in a 
cardinal direction (up/down/right/left) or at an angle - - and by varying 
the index of difficulty of the target (target size and distance). 
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CRITERION TASK: POPCORN 

OBJECTIVE: SIMULATE SUPERVISORY 

CONTROL ENVIRONMENT IN WHICH 
MULTIPLE, CONCURRENT TASKS ARE 
ACCOMPLISHED WITH AUTOMATIC 
SUBSYSTEMS 

APPLICATIONS: 

o STUDY TIME-PRESSURE ASPECTS 
OF WORKLOAD 

o COMPARE WORKLOAD PREDICTIONS 
TO ASSESSMENTS 

o PROVIDE DATA BASE FOR MODELS 
OF HUMAN PERFORMANCE 
o EXAMINE INDIVIDUAL DIFFERENCES 
(E.G.. TYPE A Vo TYPE B) 
o COGNITIVE TEST USED BY U3AF IN 
RETURNING PILOTS TO FLIGHT STATUS 



A second example of a criterion task developed at Ames is POPCORN, a 
dynamic, multi-task, supervisory control simulation. It represents 
operational environments in which decision-makers are responsible for 
actuating semi-automatic systems according to both pre-programmed and 
flexible schedules. Its name, POPCORN, reflects the appearance of groups of 
task elements waiting to be performed (they move around in a confined area 
and "pop” out when selected for performance). 

Operators decide which tasks to do and which procedures to follow based 
on their assessment of the current and projected situation, the urgency of 
specific tasks, and the reward or penalty for procrastination or failure to 
complete them. Simulated control functions provide alternative solutions to 
different circumstances. Control may be accomplished by magnetic pen and 
pad entry, mouse input, or a VOTAN voice recognition system. 

The most compelling feature of the POPCORN task is the wide variety of 
time pressure sources that can be generated, the time management strategies 
that are available, and the penalties imposed for procrastination. 
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TYPE A TYPE B 
PERSONALITY TYPE 


A recent experiment conducted jointly with SRI is one example of the 
applications in which POPCORN has been used. The objective was to provide 
empirical validation of the hypothesis that "Type A" individuals are more 
physiologically, behaviorally , and psychologically reactive to task- induced 
stressors than "Type B" individuals. It has been suggested that it is this 
differential level of reactivity that leads to the eventual development of 
cardiovascular disease associated with the "Type A" personality. 

We found very strong empirical evidence that "Type A" men with normal 
resting blood pressure levels, are significantly more reactive to different 
levels of task-induced stress than otherwise similar "Type B" males. The 
results of this study have prompted researchers at Brooks AFB to adopt 
POPCORN as one of the battery of tests to be given when returning grounded 
pilots to flight status. 
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For the remainder of this talk I will describe typical predictive 
models and measures of workload that have been developed by this program 
and the methods used in validation. 
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ASSUMPTION!: FOR WELL LEARNED TASKS. 

FUNCTIONALLY INTEGRAL ACTIVITIES PROVIDE 
THE NOMINAL LEVEL 

NOMINAL DURATION 

NOMINAL PERFORMANCE LEVELS 

HORKOAD EXPERIENCED 

ASSUMPTION 2: ADDITIONAL TASKS, CHANGES IN THE 

ENVIRONMENT, EQUIPMENT, OR PROCEDURES 

IMPAIR WHOLE OR SUBTASK PERFORMANCE 
REQUIRE ADDITIONAL TIME 
INCREASE WORKLOAD 

ASSUMPTION 3: THE INFLUENCE OF LIKELY OCCURRENCES 

DURING DIFFERENT NOMINAL ACTIVITIES CAN BE 
COMPUTED AND USED TO PREDICT NEW LOAD, LEVELS 

ASSUMPTION 4: THE RULES FOR COMBINING "EVENTS" 

WITH NOMINAL ACTIVITIES TO CREATE DIFFERENT TASKS 
MAY REFLECT: 

TASK INTEGRATION 

ADDITION 

COMPETITION 


During the past three years, we have developed a predictive model of 
pilot workload. The goal was to provide a standardize method of creating 
simulation scenarios to use in research. The initial focus of the model was 
on general aviation instrument flight (for convenience), although the model 
philosophy is being extended to helicopter operations and the space station. 
The goal was to provide a standardized format for creating simulations 
scenarios for workload and performance validation research, flight handling 
quality research, display and control evaluations and so on. 

Workload prediction must, by necessity, focus on imposed task demands 
as a starting point. We assume, that for well-learned tasks, functionally 
integrated activities that are normally performed as a unit should provide 
the basic ingredients of the model. Rather than performing a fine-grained 
analysis of the components of highly overlearned tasks (which tends to 
overestimate the workload of experienced operators), we chose to focus on a 
level of analysis that most closely represents that used by expert 
performers when describing, performing and evaluating their actions. 

The workload of these functional units - - such as specific phases of 
flight, sequences of control activities, etc - - is quantified and serves as 
the starting point for the model. Additional tasks, changes in the 
environment, equipment, procedures, or time available can be superimposed on 
these basic elements to modify the workload of the target scenario. The 
influence of these events can be computed as well, and the rules by which 
they combine with different nominal segments determined analytically, 
empirically and through expert opinions. 
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LABORATORY 
X SIMULATION 
INFLIGHT 

UNDER DEVELOPMENT 



SUBJECTIVE RATINGS 
SECONDARY TASK PERFORMANCE 
PRIMARY TASK PERFORMANCE / 
VISUAL SCAN PATTERN ! ' 




We are in the process of developing a simple "Expert 11 system for the 
selection and application of workload measures on an IBM-PC. The goal is to 
provide an interactive system whereby an individual who is not familiar with 
workload assessment, but needs to obtain information about the workload of a 
particular task or alternative pieces of equipment, can select and apply an 
appropriate measure. This system will serve to summarize and allow 
practical application of the results of our research. 

This system will assist the user in formulating the question to be 
addressed and to specify the research environment. Appropriate measures 
will be suggested and evaluated. Detailed descriptions about how to apply 
the measure will be provided along with examples and references. The system 
will be a stand-alone, user-friendly, and provide easily accessible 
information. The first application will be as a hands-on component of the 
Army MANPRINT course. 

As long as the human remains an integral element of complex, advanced 
systems, the need for standardized measures and predictors or human workload 
and performance will be required. The need for such tools is obvious both 
during the design and construction of the space station. Although the 
environment and activities to be accomplished in the space station are 
unique, the fundamental principles of human behavior and experience remain 
the same, and we are confident that the concepts and techniques that we have 
developed will provide a useful and informative tool for the development and 
operation of the space station. 
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Through extensive research, we have identified a continuum of task 
combination rules that range from: 

(1) INTEGRATION: The workload or time required to perform concurrent 

tasks approximates that of the more demanding of the components 

(2) ADDITION: The workload or time required for a complex task is equal 

to the sum of the components 


(3) COMPETITION: Task components compete for operator's attention and 

"resources" and cannot be performed within the same time interval 
There is an additional cost for switching among them and the 
cost of performing both tasks is greater than the sum of the 
parts. 
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ADJACENT TASKS ARE SIMILAR 


O0O 

ADJACENT TASKS ARE DIFFERENT 

O0D 

TRANSITIONS ARE FREQUENT 

Cfloflt* 

TRANSITIONS ARE INFREQUENT 

OSS 

NS I VENESS/ I NFLEX I B I LI TY 
NEVER AUTOMATED 


TRANSITION COST: 

TIME 

WORKLOAD 

PERFORMANCE DECREMENT 


In addition to the basic workload associated with task segments and 
additional events, there may be brief periods of relatively high workload 
associated with the transition from one task segment to another. If the 
successive tasks are similar or frequently occur together, the transitions 
may occur quickly and with low workload. If they are not, the transitions 
may be time-consuming and demanding. In addition the sheer number of 
transitions that occur during a duty period may lead to high workload levels 



DISRUPTION OF AUTOMATIC ACTIVITIES 
DURATION COST FOR SWITCHING 
OPERATORS RESPONSIVE/FLEXIBLE 


DECISION EASY 
LONG DURATION 
RECONFIGURATION DIFFICULT 


DECISION DIFFICULT 
SHORT DURATION 
RECONFIGURATION EASY 
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For each of the operational tasks to which this model is extended, a 
vehicle-specific data base is required, although the philosophy and 
structure of the model may be transferred. These nominal elements and 
additional events are entered into the computer data base and combined 
according to the appropriate algorithms dynamically by a researcher who 
wishes to create a simulation scenario of a specific duration, type, and 
workload level. The user may add and delete tasks until the predicted 
workload profile approximates the desired levels of imposed workload. The 
output of the model is a graphic representation of the predicted workload 
levels across time and a printed script to follow in conducting the 
simulation or operational test. 
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WORKLOAD 


WORKLOAD OF NOMINAL FLIGHT SEGMENTS 



The following graphs represent one such nominal and modified scenario 
developed for instrument flight for a general aviaiton aircraft. 
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The predictions of the model have been validated in a series of 
simulation experiments. A battery of converging workload assessment 
measures are imposed to test the predictions of the model. 

The first operational application of the model will be for advanced 
helicopter missions. Subsequent applications will focus on the space 
station as part of a Focused Technology Work Integration effort we will 
perform jointly with JSC. 
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’ORKLOAD PREDICTION FOR SHUTTLE 
RMS OPERATIONS 
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OBJECTIVE 

• PREDICTION OF WORKLOAD ASSOCI- 
ATED WITH OPERATOR CONTROL OF 
REMOTE MANIPULATOR ARM 

• ASSESSMENT OF ACTUAL WORKLOAD 
UNDER IG SIMULATED OPERATION 

APPROACH 

• COLLABORATIVE ARC/JSC ACTIVITY 

• FORMAL TASK DESCRIPTION 

• ANALYTIC TASK REPRESENTATION 
USING AMES MODEL 

• PART TASK TEST OF MODEL AT ARC 

• SIMULATOR VALIDATION IN RMA 
SIMULATOR AT JSC 

PAYOFF 

• GROUND VALIDATED WORKLOAD 
PREDICTION AND ASSESSMENT FOR 
RMA TASKS 

• COMPARATIVE, QUANTITATIVE AN- 
ALYSES OF NEW RMA OPERATOR 
INTERFACE-TECHNOLOGY (e.g. VOICE! 


The objective of this task is to develop and test a workload model for 
evaluation and prediction of a Space Station human operated system. The 
system selected as the first test of the model is the Remote Manipulator 
Arm. The initial focus will be on the existing RMS used in the shuttle, 
although space-station specific modifications will be incorporated as they 
are specified. 

A functional task analysis will be provided by JSC. It will be used as 
the initial data base for the prediction model. Using analytic, part-task 
simulation, and expert opinion approaches, the appropriate workload levels 
and combination rules will be determined. 

An initial test of the model will be performed at Ames, in the 
proximity opearations mockup. A simulator evaluation will be performed at 
Johnson Space Center in the RMS simulator during the second year of the 
project. This model will be used to predict the workload of alternative 
configurations and advanced RMS technology from the perspective of the human 
operator. Future applications might be to provide workload estimates as a 
feature in the existing OPSIM model developed at Ames. 

The expected product of this effort is a ground-validated workload and 
performance model that is suitable for use by contractors and Levels B and C 
personnel for the prediction and evaluation of workload and performance- 
effectiveness of human- operated Space Station systems. 
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The primary focus of this program has been the development and 
validation of of a battery of workload and performance assessment tools that 
reflect sound theoretical models of human operator performance and 
information processing. We examined existing techniques and developed 
additional ones to meet the needs of a wide variety of operational 
environments. Our goal was to provide sensitive and reliable tools and to 
disseminate information about them to make the results of our research 
widely available and practically useful. 


For each of three categories of measures - - performance, 
physiological, and subjective - - I will describe a typical technique and 
describe how it was developed and validated. 
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Early in the program, it became clear that, although human and system 
performance provided the most common motivation for workload analyses, 
performance measures themselves do not always reflect variations in operator 
workload. Within the range of their capabilities, skilled, motivated 
operators exert increasing levels of effort to accomplish increasing task 
demands. Performance degredation often occurs only after their capabilities 
are exceeded, or when they choose to maintain a consistent level of effort 
in the face of increased task demands. Subjective secondary, and 
physiological indicators of workload are more reflective of the cost of 
performance to the operator in such cases, and are able to quantify how much 
reserve capacity an operator still has when performing the task of interest. 
In addition, workload measures are able to predict future performance - - 
should task demands be increased yet farther - - while measures of 
performance are not. 

One example of a dissociation between measures of workload and 
performance is represented by a recent study completed with the POPCORN 
simulation. As time pressure was increased, performance (as measured by the 
subject* s score) dropped, as predicted. Workload levels remained constant 
however. They reflected the fact that operators maintained a consistent 
response rate in the face of increased tasks demands, and thus the cost of 
task performance - - at least as far as the operators were concerned - - 
remained constant. 
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• - -IMAhY TASK PERFORMANCE: 
FREQUENCY OF COMMUNICATIONS 


OBJECTIVE: 

EVALUATE SENSITIVITY OF DIFFERENT 
MEASURES TO NORMAL WORKLOAD 
VARIATIONS IN FLIGHT 
APPROACH: 

OBTAIN DIFFERENT MEASURES DURING 
11 ROUTINE MISSIONS OF THE NASA 
KUIPER AIRBORNE OBSERVATORY (C-141) 
RESULTS: 

RATED WORKLOAD AND COMMUNICATIONS 
FREQUENCIES VARIED SIGNIFICANTLY 
ACCROSS FLIGHT SEGMENTS 

RATED WORKLOAD 



COMMUNICATIONS/MINUTE 


20 


111 


FLIGHT SEGMENT 


FLIGHT SEGMENT 


Selected measures of performance may covary with operator workload. In 
a study that we conducted in the Kuiper Airborne Observatory, we found that 
the rate of communications activities provided a convenient and sensitive 
measure of the overall levels of workload imposed on the flight crewmembers. 

In addition, we have found that specific types of communications are 
associated with different levels of workload. A post hoc communiat ions 
analysis can provide a sensitive workload evaluation in a many of 
environments, using data that is readily available in most operational 
environments . 
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COMMUNICATIONS ANALYSIS: MEASURES OF CREW COORDINATION 

AND DECISION MAKING 


OBJECTIVE: 

ANALYZE FLIGHT DECK AND ATC COMMUNICATIONS TO ASSESS AIRCREW DYNAMICS 
COMMUNICATIONS COMPETENCY. AMD AIRCRAFT MANAGEMENT 


APPROACH: 

o CONDUCT SIMULATIONS IN B-707 
SIMULATOR 

o OBTAIN POST-FLIGHT EVALUATIONS BY: 

(1) CREWMEMBERS 

(2) EXPERTS IN LINGUISTIC 
AND SEMANTIC ANALYSIS 

(3) EXPERTS IN FLIGHT SAFETY 

RESULTS: 

o CREWS DIFFERED IN COMMUNICATIONS 
COMPETENCY AND LEADERSHIP ROLES 
0 CREW COORDINATION AFFECTED DECISION 
MAKING AND AIRCRAFT MANAGEMENT 


- 






" i 'V 






Another facet of communications that we have investigated is the role 
of flight deck communications in aircrew organization and coordination. In 
a recent simulation of transport operations, we found that crews differed in 
communications competency. Communications analyses provided a sensitive 
measure of leadership and crew coordination - - factors that play important 
roles in the safety and efficiency of aircrew performance. Crew 
coordination affected decision making behavior and aircraft management. 

The primary goal of this part of the program is to develop a training 
program to improve crew communications competency, corrdination and 
leadership . 
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PHYSIOLOGICAL MEASURES: 
EXAMPLES 

o 

MEASURES OF MENTAL AND 
PROCESSING 

PERCEPTUAL 


X 

EVOKED CORTICAL POTENTIALS 




X 

EYE POINT OF REGAHD 



o 

MEASURES OF EMOTIONAL 
ACTIVATION 

AND 

PHYSICAL 


X 

HEART RATE AND VARIABILITY 

X 

MUSCLE TENSION 


X 

BLOOD PRESSURE 

X 

VOCAL STRESS 


X 

GALVANIC SKIN RESPONSE 

X 

PUPIL SIZE 


X 

RESPIRATION RATE 




We have investigated a number of physiological measures of workload. 
Several measures provide relatively specific indicators of mental and 
perceptual processing - - such as auditory evoked cortical potentials. In 
addition, we have examined a number of measures that reflect more general 
levels of activation, such as heart rate, and pupil size. The advantage of 
physiological measures is that they are unobtrusive, do not interfer with 
primary task performance, and they provide common, objective measures across 
a variety of tasks. 
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AVERAGE HEART RATH 
r <BEAT8-'HIH) 


NASA 0141 

WORKLOAD 

SIUDY 


LEFT SLAT 


PILOT KITING OF 
OUE&ALL MOH&LO&D 




©EGMEHT OP FLIGHT 


HEARTRATE 


REFLECTS STRESS, NOT BUSYNESS 

IS MORE SENSITIVE TO STRESS THAN SUBJECTIVE RATINGS 


The research we have conducted in evaluating heart rate and heart rate 
variablity is one example of this area of research. Heart rate provides a 

convenient and nonintrusive indicator of the overall level of activation of 
an operator. It is less likely to reflect more subtle changes in workload 
associated with different levels of mental activities, however. In the 
study that I mentioned earlier, we obtained measures of pilot heart rate 
during 11, eight-hour routine missions of the Kuiper Airborne Observatory 
using the portable Vitalog physiological recording unit. 

The heart rate profiles of the pilot- flying , reflected 
the expected peaks during take-off and landing. The profiles of the pilots- 
not-f lying reflected no significant changes, however. These results, in 
agreement with earlier studies, suggest that heart rate reflects 
responsibility and stress, rather than mental workload. 

These data are particularly interesting because the test pilots who 
participated in the study were qualified in both positions, and the same 
pilots are represented in the data for both. The pilots experienced and 
reported apparently similar levels of subjective workload throughout the 
flight, but the heart rates suggested that there were differences in the 
physiological consequences of performing the duties required by the two 
positions . 

In other studies, we have found that heart rate is quite insensitive to 
the variations in levels of workload imposed by a wide variety of laboratory 
tasks unless rather heavy physical effort is involved. 

These data again point out the need for multiple, 
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converging measures 



of workload to obtain the most complete picture possible of the impact of 
performing a task on the operator. 

We are focusing most of our research efforts in the area of heart rate 
variability. In particular, we have evaluated the power in the .1 Hz range 
of the frequency spectrum of the beat-to-beat intervals as a very promising 
measure. There is considerable evidence that this measures provides a 
sensitive indicator of different levels of mental workload. The typical 
finding is that heart rate variability (and the power in the .1 Hz region) 
decrease as mental workload is increased. A n black box 11 has been developed 
to obtain and process this measure automatically online. 
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SUBJECTIVE RATINGS: 
ISSUES 

wsES3 3ss sa ^sB3BBa m^Easssgm m 


PROVIDE SIGNIFICANT SOURCE OF INFORMATION 

MAY TAP THE ESSENCE OF MENTAL WORKLOAD 

REFLECT SUBSET OF INFORMATION AVAILABLE 
DURING TASK PERFORMANCE 

- RESULTS OF INFORMATION PROCESSING 

- MEMORIES 

- OVERT BEHAVIOR 

- FEELINGS 

INDIVIDUAL DIFFERENCES IN DEFINITION AND 
AND EXPERIENCE 

NO MENTAL REFERENCE SCALE FOR "WORKLOAD" 

BEST TO COMPARE SHARED QUALITIES AND 
SIMILAR ACTIVITIES 

CALIBRATION OF RATERS 

TIMING 

- ON-LINE ys RETROSPECTIVE 

- PRIMACY/RECENCY OR ODDBALL EFFECTS 

PSYCHOMETRIC CONSIDERATIONS 

- EQUALITY OF INTERVALS 

- NO "ZERO" POINT OR "MAXIMUM" 


Considerable effort has been devoted to understanding and measuring the 
subjective workload experiences of operators, as this is the most convenient 
and practically useful measure. In addition, it is the measure against 
which most other measures are calibrated. We have found that subjective 
ratings provide a significant source of information, come closest to tapping 
the essence of mental workload, and provide the most direct indicator about 
the subjective impact of a task on operators. 

People often generate evaluations about the workload of ongoing 
experience, however they rarely quantify or remember such experiences. 
Thus, experiencing workload is unique to experimental situations, although 
the requirement to verbalize, remember or quantify such experiences may not 
be a commonplace activity. The goal of our research has been to determine 
what factors influence such subjective experiences (and which ones do not) 
and to develop a valid, sensitive, and reliable measure of them. 
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THE TYPES OF EXPERIMENTAL TASKS INCLUDED IN THE WORKLOAD RATING SCALE 

DEVELOPMENT EFFORT 


0 SIMPLE. COGNITIVELY-LOADING TASKS 

CHOICE REACTION TIME. MEMORY SEARCH. MENTAL ARITHMETIC. 

MENTAL ROTATION. PATTERN MATCH 

o SIMPLE. MANUALLY-LOADING TASKS 
ONE AND TWO AXIS TRACKING 

0 CONCURRENT. INDEPENDENT DUAL-TASKS 

TRACKING + MEMORY SEARCH. MENTAL ROTATION 

o SERIAL. INTEGRATED "FITTSBERG" TASKS 

TARGET ACQUISITION + MEMORY SEARCH. MENTAL ARITHMETIC. RHYMING. 

PATTERN MATCH. PREDICTION. TIME ESTIMATION 

o COMPLEX SUPERVISORY CONTROL SIMULATIONS ("POPCORN") 

0 PART-TASK AND FULL-MISSION AIRCRAFT SIMULATIONS 


During the past three years, we have conducted a series of 25 
experiments in which a multi-dimensional battery of bipolar rating scales 
were presented to subjects following a variety of tasks. For 15 of these 
experiments, the ratings, and individual definitions of workload were 
combined into a data base and a number of global analyses were performed. 

The objective was to determine: 

(1) What factors are sensitive to workload differences between 
different types of tasks 

(2) What factors are sensitive to workload differences within tasks 

(3) What factors are included in the workload definitions of most 
individuals 

(4) What is the appropriate scale format 

The primary problems that we encountered in this effort were: 

(1) There is no objective standard against which workload ratings can 
be compared 

(2) The workload of a task is not uniquely defined by its objective 
demands but represents the behaviors and psychological responses 
of individual subjects as well 

(3) Different individuals may adopt different references activities 
and have diffferent personal definitions of workload 

We organized the experimental tasks into six categories. These tasks 
ranged from simple, cognitively loading tasks to complex aircraft 
simulations. Several thousand data points were included in each category. 
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We found that different individuals consider different variables in 
formulating workload ratings. Thus, one person* s overall workload rating 
might reflect the level of time pressure experienced while another’s might 
reflect the level of cognitive effort exerted or their apprarent failure to 
accomplish the task requirements. People are generally unaware of the 
fuzziness of their definitions, however, they are able to express their 
biases when asked to do so. 
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We found, that by weighting the bipolar ratings obtained on the 
component scales by the subjective importance of each factor to each 
subject, and by averaging these weighted ratings, we were able to obtain a 
significant reduction in between-sub ject variability in a summary estimate 
of overall workload. 


These summary scores reflected the same workload levels indicated by 
overall workload ratings, but with a 25-507 reduction in variability. 
However, the sensititvity of the summary measure to experimental 
manipulations was not significantly enhanced. 


71 

















Since workload represents a collection of attributes, the sources of 
workload may vary from one activity to the next as a result of the 
requirements, equipment, and environment. Thus, the workload of one task or 
task segment might be created by very heavy physical demands, while that of 
another by the level of time pressure or danger. 

Although individuals may define workload differently, they are, none- 
theless responsive to the specific sources of loading imposed by a task. 
Since the subjective experience of workload emerges from the interaction 
between objective task requirements and an individuals response to them, we 
found that it was critically important to determine the subjective 
importance of specific factors in creating the workload of a specific 
activity (as well as the magnitudes of those factors) to develop a sensitive 
and accurate multi-dimensional rating of overall workload. 
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We found that at least six factors are necessary to discriminate 
between workload levels within and between tasks. They are: 

Task related: 

Temporal Demands, Physical Demands, and Mental Demands 
Sub ject- related: 

Own Performance, Frustration, and Effort. 

Each of these scales alone provides useful, diagnostic, and often 
independent information about the sources of workload and the experiences of 
operators. By combining these individual scale values, weighted to reflect 
their importance in creating the level of workload imposed by a specific 
task, a global indicator of overall workload can be derived that is less 
variable between subjects and more sensitive to experimental manipulations 
than are existing rating technqiues. 
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A priori workload weights, which form the basis for several popular 
techniques, do not reflect the objective contributions of specific factors 
to the workload of a specific task. The model presented in this figure 
represents the conceptual framework of the rating technique that we 
developed. Objective demands are imposed on an operator, which are 
translated into psychological representations. These invoke behavioral and 
psychological responses from an operator. A weighted combination of the 
relevant factors - - both objective and subjective - - are integrated into a 
subjective experience of workload that may be translated in to a numeric or 
verbal evaluation. The key element of this model is that the integration 
represents a weighted combination of factors. The weights reflect the 
objective and subjective importance of the factors to the structure of that 
task and the ratings reflect the psychological magnitudes of each factors 
during that activity. 

The bipolar rating scale that we propose is two dimensional: 
evaluations of the magnitude as well as the importance of each of six 
factors are obtained from subjects following specific tasks or task 
segments. The combined weighted average of the six factors provides a 
sensitive and stable measure of overall workload. 
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With this measure, as with all of the others, validation is 
accomplished in a variety of environments. Each measure is tested against 
criterion tasks that impose known, well-controlled levels of workload. 
Promising measures are then tested in part-task simulations within our lab. 
Finally, many measures have been applied - - piggy-back - - on a variety of 
operational activities to provide "real-world" validation. 
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VALIDATION OF NASA WORKLOAD ASSESSMENT MEASUREMENT BATTERY 


OBJECTIVE: DETERMINE THE SENSITIVITY AND OPERATIONAL VALIDITY OF THE 
WORKLOAD MEASURES DEVELOPED AT NASA-AMES 

APPROACH: CONSTRUCT SCENARIOS WITH WORKLOAD PREDICTIVE MODEL 

PERFORM FLIGHTS IN B-727 SIMULATOR AND SH-3 HELICOPTER 
COMPARE MODEL PREDICTIONS TO EMPIRICAL RESULTS 

MEASURES: PERFORMANCE (COMMUNICATIONS. ERRORS. CREW COORDINATION. 

CONTROL VARIABILITY, SECONDARY TASKS) 

PHYSIOLOGICAL (HEART RATE/VARIABILITY. EYE BLINK RATE/TIMING. 

SCAN PATTERN. AUDITORY EVOKED CORTICAL POTENTIALS) 
SUBJECTIVE (NASA MULTI-DIMENSIONAL SCALE. REFERENCE TASK 
COMPARISON. MODIFIED COOPER-HARPER SCALE) 



The final validation effort for our workload-assessment battery will 
be accomplished within the next year. We plan to conduct at least two full- 
mission studies in which all of the most promising measures will be applied 
in realistic environments. The test scenarios will be created with the 
workload predictive model. Two environments have been selected for these 
studies : 


(1) The MVSRF 727 motion-base simulator 

(2) A Sea-king (SH-2) helicopter. 

Our goal is to provide as complete and as operationally relevant a 
validation of the measures as possible in a well-controlled and realistic 
series of flights. 

Concurrent with this effort, the predictive model for Space Station 
application will continue, and it will be validated at JSC in 1987. 
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SPATIAL COGNITION 


Mary Kister Kaiser 
Reger Remington 

NASA Ames Research Center 


SPATIAL COGNITION is the ability to reason about geometric 
relationships in the real (or a metaphorical) world based on one 
or more internal representations of those relationships. The 
study of spatial cognition is concerned with the representation 
of spatial knowledge, and our ability to manipulate these 
representations to solve spatial problems. Spatial cognition is 
utilized most critically when direct perceptual cues are absent 
or impoverished. 

Our presentation provides examples of how human spatial 
cognitive abilities impacts on three areas of space station 
operator performance: orientation, path planning, and data base 
management. A videotape provides demonstrations of relevant 
phenomena (e.g. the importance of orientation on recognition of 
complex, configural forms). The following readings are 
suggested as entries into the psychological literature on spatial 
cognition: 

Olson, D. R. & Bialystok, E. (1983). Spatial Cognition . Hillsdale, 
New Jersey: Lawrence Erlbaum Associates, Inc. 

Rock, I. (1984). Perception . New York: Scientific American 
Books, Inc. 

Shepard, R. N. & Cooper, L. A. (1982). Mental Images and their 
Transformations . Cambridge, MA: The MIT Press. 
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AREAS IN WHICH SPATIAL COGNITION 
IMPACTS PERFORMANCE: 

* ORIENTATION 

* PATH PLANNING 

* DATA BASE MANAGEMENT 
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SPECIAL CONCERNS OF SPACE STATION 


* ABSENCE OF VESTIBULAR CUES 

* MULTIPLE FRAMES OF REFERENCE 

* ADDITIONAL DEGREES OF FREEDOM 

* COMPLEX, NON-INTUITIVE DYNAMICS 



ORIENTATION ONBOARD THE SPACE STATION: A CHALLENGE TO 
SPATIAL COGNITION 



FRH H yr r yr r>[ ICAL 



TELEOPERATOR VERTICAL 


MULTIPLE CAMERA ANGLES 
(POINTS-OF-VIEW) 



STATION LOCAL VERTICAL 
EARTH DOWN VERTICAL 


SAIELLITE VERTICAL 




SPATIAL TRANSFORMATIONS INVOLVE TIME-DEPENDENT MENTAL 
OPERATIONS 


MENTAL ROTATION 






PAIR A PAIR B 

"SAME" "DIFFERENT" 



Angle of Rototion ( degrees ) 
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OBSERVERS IMPOSE KINEMATIC AND KINETIC CONSTRAINTS ON AMBIGUOUS 
VISUAL DISPLAYS 


CO 

to 









THREE POSSIBLE STRUCTURES FOR A DATA “ENVIRONMENT 


Large Single Plane 


a 



Single Plane with 
Stacked “Windows” 
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VIRTUAL INTERFACE ENVIRONMENT 


Scott S. Fisher 

NASA Ames Research Center 
Moffett Field, California 94035 


Abstract 

A head-mounted, wide-angle, stereoscopic display system controlled by operator position, voice 
and gesture is under development for use as a multipurpose interface environment. Initial appli- 
cations of the system are in telerobotics, data-management and human factors research. System 
configuration and research directions are described. 


1. Objective 

The objective of this research effort is to develop a multisensory, interactive display system in 
which a user can virtually explore a 360-degree synthesized or remotely sensed environment and 
viscerally interact with its components. This work is done in the context of developing a mul- 
tipurpose operator interface “environment ’ to facilitate natural interaction with: 

- Complex operational tasks such as control of remotely operated robotic devices and vehicles that 
require a sufficient quantity and quality of sensory feedback to approximate actual presence at 
the task site. 

- Large-scale integrated information systems in which data manipulation, storage and retrieval 
and system monitoring tasks can be spatially organized. 

An additional research objective includes use of this display system to synthesize interactive test 
environments for aerospace human factors research in such areas as: spatial habatability research, 
rapid prototyping of display and workstation configurations, research on effective transfer of spa- 
tial information, and spatial cognition research on multisensory integration. 


2. System Configuration 

The virtual environment display system consists of: a wide-angle stereoscopic display unit, glove- 
like devices for multiple degree-of-freedom tactile input, connected speech recognition technology, 
speech-synthesis technology, gesture tracking devices, and computer graphic and video image gen- 
eration equipment. 

The present display unit is helmet-mounted and consists of monochromatic liquid crystal display 
screens presented to each eye of the user through wide-angle optics. The effective field of view for 
each eye is 120 degrees for horizontal and vertical. Imagery displayed on the screen is generated 
by computer, remote video sources or a combination of input media. Head motion of the user is 
tracked by a sensor mounted on the helmet and the derived position and orientation data is used 
to update the displayed stereo images in response to the users activity. As a result the displayed 
imagery can appear to completely surround the user in 3-space. 


Presented at the Space Station Human Factors Research Review, December 3 - December 6, 1985. NASA Ames 
Research Center, Moffett Field California, 94035. 
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To interact with the displayed three dimensional environment, the user wears lightweight glove- 
like devices that transmit data-records of arm, hand and finger shape and position to a host com- 
puter. In coordination with connected speech recognition technology, this information is used to 
effect indicated gestures in the synthesized or remote environment. Current examples of research 
in voice and gesture mediated interaction is the control of robotic arms and end-effectors, and 
associated control of auxiliary camera positions. Similarly, in a data management environment, 
windows of information or virtual control panels are positioned, sized and activated in 3-space. 

Current experimental research includes system calibration for orthoscopic image display, evalua- 
tion of operator performance in teleoperation placement tasks, and analysis of perceived localiza- 
tion of synthesized sound sources in 3-space. 


3. Conclusions 


Unlike most contemporary 360-degree visual simulation environments, the virtual environment 
display system does not make use of large, expensive, special purpose projection configurations; 
The described system is portable and low-cost without large space and equipment requirements. 
Unlike other research efforts in head-mounted displays, this system is unique in presenting a 
stereoscopic image that fills the user’s field of view 7 completely and in its configuration with 
speech and tactile input technology. 

As a research tool, the virtual environment display system follows many research efforts to 
develop operator control stations for teleoperation and telepresence but has a unique configuration 
to investigate natural, multisensory interaction with complex operational tasks. As an interface 
for data management tasks, this system is a continuation of recent research in multimodal, 
natural input technology and concentrates on development of a true three dimensional data space 
interface that can be easily reconfigured for idiosyncratic users. 
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FAULT DIAGNOSIS IN ORBITAL REFUELING OPERATIONS 1 


Guy A. Bov 2 
ONERA-CERT-DERA 3 
2, avenue Edouard Belin 
31055 Toulouse Cedex 
F ranee 


Abstract 

Usually, operation manuals are provided for helping astronauts during space 
operations. These manuals include normal and malfunction procedures. 
Transferring operation manual knowledge into a computerized form is not a 
trivial task. This knowledge is generally written by designers or operation 
engineers, and is often quite different from the user logic. The latter is usually a 
"compiled" version of the former. Experiments are in progress to assess the user 
logic. HORSES (Human - Orbital Refueling System - Expert System) is an 
attempt to include both of these logics in the same tool. It is designed to assist 
astronauts during monitoring and diagnosis tasks. Basically, HORSES includes a 
situation recognition level coupled to an analytical diagnoser, and a meta-level 
working on both of the previous levels. HORSES is a good tool for modeling task 
models and is also more broadly useful for knowledge design. 

Keywords: On-Line Expert System, Man-Machine Interactions, Process Control, 
Diagnosis System, Knowledge Design, Task Models, Situation Recognition. 


1 This paper will be presented at the Space Station Human Factors Research Review, NASA Ames Research 
Center, December 3 to December 6, 1985. 

2 This work was completed when the author was a Research Associate at NASA-Ames Research Center, Aero-Space 
Human Factors Research Division, Mail Stop 239-3, Moffett Field, CA 94035, U.S.A.. 

3 ONERA: Office National d’Etudes et de Recherches Aerospatiales ; CERT: Centre d’Etudes et de Recherches de 
Toulouse ; DERA: Departement d’Etudes et de Recherches en Automatique. 
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Fault Diagnosis 

In Orbital Refueling Operations 


Guy A. Boy 


Space Station Human Factors Research Review 
NASA Ames Research Center 
December 6, 1986 
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1 . Problem Definition 


Human-Machine Interactions 
in Normal and Abnormal Situations 

□ Understanding the HMI Logic 

A Human Operator Model 
A System Logic vs. User Logic 


□ Need for, and Limitations of Al Tools in 
System Operations 

A An Example : The ORS 


□ Building A User's Guide Expert System 

A Operation Manual 
A An Al Tool, why ? (modularity, 

flexibility, ...) 

A Human vs. Automatic Diagnosis 
A Human-Expert-System Interactions 
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LANDSAT-D REFUELING 



Landsat-D will utilize the ORS equipment and procedures for 
propellant replenishment. 
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ORS PROTOTYPE FLOW SCHEMATIC 
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SITUATION 
PATTERN- 1 



SITUATIONAL AND ANALYTICAL PROCESSES 
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2. Building a User’s Guide Expert System 


□ Goals 

A Optimal Level of Automation 
A Explanation 
A Easy-to-Use Interface 


□ Methods 

A Modelling Approach 
A Human Factors Studies 
A Triangular Interactions 

□ Tasks 

A Buiding an Expert System 
A Experiments 
A Theoretical Studies 

□ Product 

A Tool to Design Procedures 
A Diagnosis Aid 
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Human - ORS - Expert System 


□ Processor 

A Situation Recognition ( Monitoring ) 

A Diagnosis Inference Engine ( 2 levels ) 

□ Knowledge Base 

A Context Rules 
A Regular Rules 
A Meta Rules 
A Predicates 
A Tolerance Functions 
A Objects 

□ Interfaces 

A User Interface ( Question-Answer, Menus ) 
A ORS Interface ( Fact Filter, Fuzzy Models ) 
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HORSES BACKGROUND 


MESSAGE 

(ONERA / Airbus Industrie) 

(Certification, Workload & Performance Analyses) 

SEAGOS 

(ONERA / Matra) 

(Satellite Malfunction Procedures) 

HORSES 
(NASA / ONERA) 

(ORS Malfunction Procedures) 
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HORSES Current Version 


□ Working in Lisp on MASSCOMP 

□ Connected to an ORS Fortran Simulation 

□ Graphic Interface ( Windows, Color ) 



THE 



STANDARD 

SWITCH 

PANEL 


NDOWS 
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Further Studies 


□ Experiments on Man-Machine Interactions 

A Level 0 ( Paper Manual ) 

A Level 1 ( Expert System Guides and Advises ) 
A Level 2 ( Automatic Diagnosis, Explanation ) 

□ Situation Recognition 

A Experiments on Qualitative Models 
A Fuzzy Sets Approach 

□ Explanation 

A Information on Time and 
at the Appropriate Level of Detail 
A Graphic Displays 

□ Knowledge Editor 

A Consistency 
A Graphic Displays 
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Operator Assistant 


• COMPUTERIZED OPERATION MANUAL 

• SITUATION RECOGNITION SYSTEM 

• COOPERATIVE DIAGNOSIS ADVISOR 

• DIFFERENT LEVELS OF AUTOMATION 

• DYNAMIC AND INTERACTIVE 
• EASY-TO-USE 
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Tool for Implementing 
Task Models 


• KNOWLEDGE DESIGN 
• KNOWLEDGE PROCESSING 
• VISUAL THINKING 
• GRAPHICAL INTERFACE 
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ERROR-TOLERANCE AND PROCEDURE AIDS 


EVERETT PALMER 

• Approaches to the Problem of Human Error. 

• The Concept of Error Tolerance 

• Research Focus - Past and Present 

• A Conceptual Design for an Error Tolerant 

System 
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Human Error 


The Problem. 

• Human error is the primary cause of 60 to 90% of major 
accidents. 

Standard Approaches to Reducing Human Error. 

• Procedures 

• Interlocks 

• Warnings 

• Automation 

Possible Result of these Approaches, if Taken 
Too Far. 

• Operator Out of the Loop 

• Reduced Ability to Cope with the Unexpected 
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Why Have a Human in the System? 


■ A Primary Reason is to Cope with the Unexpected by 

Innovating. 

> Flexible, Adaptive Information Processors 

> Recognize Patterns 

> Make Associative Leaps 

> Persevere in Ill-Structured, Ambiguous Environments 

■ The Cost of these Benefits 

> Human Errors 

• Apparent Dilemma : 

> Human Innovation vs. Human Error 

■ Errors are Not Inhehently Unacceptable 

• Errors can be Tolerated if the Consequences of Error 

Can be Control ed 


Aeronautics Technology - Possibilities for 2000 
Aeronautics and Space Engineering Board 
National Research Council 
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Error Tolerance vs. Error Reduction 

• Error Tolerance 

> Focus on the Observable End of the Causal Chain 

Leading to the Error. 

> Focus on Explanations and Compensations for 

Particular Problems that Have Occured. 

* Error Reduction 

> Emphasizes Upstream Aspects of the Problem. 

> Concerned with Probabilities of All Potential 

Problems That Might Occur. 
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Research Focus: Procedure Error Detection 


Motivation: 

• Procedures are an essential part of the operation of any 
complex dynamic system. 

■ Error detection is the first requirement of an error tolerant 
system. 


6oal: 

■ To design systems that understand the goals and 
plans of their human operators and that can use this 
knowledge to detect and inform the operators of 
possible human errors. 
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Research Focus: Procedure Error Detection 


Objective: 

• Develop a software system that can detect procedural 

errors in the operation of aero-space systems. 

Approach: 

• Base the system on script based Al programs that 

understand human actions in simple stories. 

• Develop a hierarchial script based program to detect 

procedural errors in a general aviation simulator. 

• Extend the program to detect errors in a full mission 

B-727 aircraft simulator. 

■ Incorporate the program concepts into a "SMART 
CHECKLIST" for the Advanced Cockpit Flight 
Simulator" and a space shuttle payload. 
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Background Research: 

■ Georgia Tech Grant 

(Bill and Sandra Rouse, John Hammer) 

■ Experimental Evaluation of Electronic Checklists 

> Full Mission General Aviation Flight Simulator 

> Checklist Display Conditions 

» Hard Copy - Paper 
» Aided Soft Copy - CRT 

> Results 

» Hard Copy: 1 9% faster 
» Aided Soft Copy: 7.5 times fewer errors 
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Automatic Procedure Error Detection 

■ Initial Objective: 

> Data Analysis Tool 

> Detect Errors: 

Omitted 

Incorrect 

Out-of-Order 

■ Ultimate Objective: 

> Real Time Feedback to the Crew 

• Infer Current Procedure - Track Context 

• Story Understanding -> Crew Action Understanding 

■ Scripts: 

> A script describes the actors and actions that 

can be expected to occur in a given situation. 

• Hierarchical Tree Structure 

• A model of the human for the system. 
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Automatic Procedure Error Detection 



Tree of steps, procedures, phases and flight. 



State transitions for tree nodes 


122 






Current Projects at NASA Ames 

1) Procedure Error Detection in a 727 Simulator 

2) "Smart Checklist" 

* Applications: 

> The Orbiting Refueling System - the ORS 

> Space Shuttle Ground Control - the INCO Position 

> The Advanced Cockpit Flight Simulator 

* Operational Modes: 

> Browser 

> Editor 

> Monitored (Management by Exception) 

> Automatic (Incremental Automation) 

* Features: 

> Call up relevant schematic display 

> Highlight relevant information on the schematic 

> Show desired or expected state 

> Show actual state 

> Show if procedure step is satisfied 

* Goal Sharing 

* Goal Inference vs. Goal Communication. 
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An Example of the "Smart Checklist" for the ORS 


ORS 

TRANSFER 1 

SM 223 ORS SUMMARY 

DONE 

IN-PROGRESS 

IN-PROGRESS 

J_ Configuration 

Checks 


IN-PROGRESS 

✓ ors“htra 

OFF 

OFF 

DONE 

l HRT B 

AUTO 

OFF 

IN-PROGRESS 

7 SW ENA 

INH 

OFF 


7 VDE PWR 

OFF 

OFF 


-/TANK 1 PI 

260+ 15 psl 

254 


/ TANK 2 P2 

130 + 15 psi 

146 


7 TANK 2 T2 

< 80 degF 

86 


✓ V3 

OP 

OP 


-/ V7 

OP 

OP 


J VI 

CL 

CL 


y V4 

CL 

CL 


y vs 

CL 

OP 
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A Conceptual Design for an Error-Tolerant 
System 

• System Functions: 

> Error Identification 

> Error Classification 
» Slips vs Mistakes 

> Error Remediation 

• Design Philosophy: 

> Non-lntrusive Monitoring 

> Incremental Intervention 

> Dependence without Loss of Control 

• Design Issues: 

> How and When Should the Crew be Informed of 
Their Errors? 

> Can the System be Supportive without Also Being a Nag? 

> If there is Disagreement, Who is In Charge? 
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Automatic Procedure Error Detection 


GOAL: 


• TO DESIGN SYSTEMS THAT UNDERSTAND THE GOALS 

AND PLANS OF THEIR HUMAN OPERATORS AND THAT CAN USE THIS 
KNOWLEDGE TO DETECT AND INFORM THE OPERATORS OF 
POSSIBLE HUMAN ERRORS. 

• A SCRIPT BASED CREW ACTION 

UNDERSTANDING PROGRAM 

• MODELS PROCEDURAL ACTIONS 

• INFERS PILOT'S CURRENT 

GOALS, PLANS AND PROCEDURES 

• DETECTS OMITTED, INCORRECT AND 

OUT-OF-ORDER ACTIONS 



APPLICATIONS 

• FULL MISSION DATA ANALYSIS 

• ERROR TOLERANT COCKPITS 

• ERROR TOLERANT WORK STATIONS 
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SPACE STATION HUMAN FACTORS RESEARCH REVIEW 


December 6, 1985 


INHOUSE ADVANCED DEVELOPMENT AND RESEARCH 


Chaired by: Trieve Tanner 


I wish to extend my acknowledgements to the following people: 

• all of the speakers 

• chairpeople: Vic Vykukal, Yvonne Clearwater, and Marc Cohen 

• various support people, whom I have already thanked 

• Marc Cohen again for coordinating the overall planning 

• the AV support people 

• Kathleen Connell, who coordinated the logistics 
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